New idea exchange ideas in Databricks Community

Add a Knowledge tab to Unity Catalog assets: grounded Q&A, steward-confirmed answers, and AI-maintai

fddemarco — Sat, 03 Oct 2026 15:16:02 GMT

Summary

Unity Catalog documents what data is supposed to be, but not what users have learned by using it. I'm proposing a persistent Q&A and knowledge layer on catalog assets, so that questions get answered where people discover the data and the answers stay with it.

Problem

Practical knowledge lives outside the catalog. Caveats such as "this table lags by two days", "this ID changed in the 2024 migration" or "don't use this field for revenue" sit in Slack, Teams, tickets and people's heads. The next analyst never finds them.
The same questions get asked repeatedly. A question is asked privately, answered, and lost, so it gets asked again six months later.
Stewards are a scarce resource. Many consumers don't know who owns an asset or whether they can contact them. Stewards who are reachable get overwhelmed with repeat questions.
Governance teams get no signal on documentation quality. They can't see which fields users demonstrably don't understand.

Proposed solution

Add a Knowledge tab to catalog assets (tables and columns first, then views, schemas, models and others), alongside Overview, Lineage and Quality.

Ask: Users ask questions in context. The AI answers using the catalog's own context: metadata, comments, tags, lineage, quality information, usage, and prior Q&A on the asset.
Rate: Users can mark answers Helpful, Incorrect, or Needs clarification. Conflicting feedback is shown, not hidden.
Escalate: When AI confidence is low, the question can be routed to the community, a data steward, or the data owner. Their answer becomes persistent knowledge on the asset.
Confirm: Authorized users can mark answers as confirmed or pin them as authoritative.
Summarize: AI maintains a concise knowledge summary per asset (known caveats, open questions, community confidence), with every claim linked to its source discussion.
Find gaps: Surface fields that generate many repeated questions (for example, "47 questions about customer_status in 90 days") so governance teams know what to document.

Trust and governance requirements

Provenance labels. Every item is labeled as one of: official metadata, steward-confirmed, community, AI-generated, or unverified.
Permissions. Discussions inherit the access controls of the underlying asset. Column-level discussions must not expose content to users who can't see that column.
Freshness. Confirmed answers are flagged for review when the asset's schema, definition or lineage changes, so stale confirmations don't create false assurance.
Moderation. Stewards and owners can edit, retract, or archive answers.
Auditability. AI answers cite the sources they were built from.

Expected benefits

Data consumers: faster answers, fewer wrong assumptions, less dependence on individual experts.
Stewards: answer once and help many; fewer repetitive questions; questions arrive with context.
Governance teams: scalable stewardship, durable institutional knowledge, and evidence-based prioritization of documentation work.

The aim is to extend Unity Catalog from a system of record for metadata into a system of record for what an organization has learned about its data.

Admin Setting to Pause All Jobs for Specific User Groups During Deployments

jaswanth333 — Fri, 06 Mar 2026 19:15:13 GMT

Problem

In development environments, scheduled jobs may continue running during deployments if users forget to disable them. This leads to extra compute costs, job failures, and avoidable pipeline conflicts.

Proposed Feature

Add an Admin Console → User Groups option that lets admins enable a setting to pause all jobs for a user group during deployments.

Benefits

Reduces unnecessary compute costs
Prevents failures during updates
Provides centralized governance
Ensures predictable and safe deployment windows

Go to Current Running Cell” Button in Databricks Notebook Editor

jaswanth333 — Fri, 06 Mar 2026 19:08:57 GMT

Team,

Please add a button in the notebook editor UI that automatically scrolls the view to the currently running cell. This would help users:

Navigate long notebooks more efficiently
Reduce manual scrolling
Improve debugging and workflow speed
Enhance the user experience during long-running jobs or complex workflows

Disable Marking of Own Post as Solution

TheOC — Mon, 01 Sep 2025 14:33:05 GMT

I've noticed a few members of the Community marking their own posts as solutions. It's not exactly a major issue but I think it dilutes/diminishes others who are trying to work on achieving authentic solutions and likes.

I think this could easily be solved by not allowing people to solve their own posts as solutions. I do appreciate there may be other ways of people abusing the system but this would at least ensure that solutions are somewhat authentic.

Ramping up Databricks Badges / Community Profiles

BS_THE_ANALYST — Fri, 22 Aug 2025 09:37:14 GMT

Hey everyone,

If I click on a user's profile, I'd love to see the databricks certifications they've achieved and whether they're still valid or have expired. It's a great way to use the community profile as part of a portfolio.
Could we also get a profile badge for each certification we achieve?
Why do we seem to have a cap at 75 for badge categories i.e. 75 questions answered? Can we increase the number of badges within these categories but have bigger jumps at the later stages i.e. 75->125->200 etc.

Kind regards,
Ben

Automatically @ the user I'm replying to

TheOC — Tue, 19 Aug 2025 15:27:25 GMT

Hi,
It would be really useful if on the Community, when I'm replying to another user, it automatically starts the message with an @user. This would make it much clearer that I'm directly replying to that user, and increase the number of solutions on the Community - as per the previous post by Databricks.

Thanks!

'Remember me' option on the community

TheOC — Tue, 19 Aug 2025 14:45:44 GMT

Hello,
Maybe similar to my Captcha idea: But it would be great if the Community had a 'remember me' / 'stay logged in' option so that users don't have to login daily.

This would make the Community much more fluid to use as a regular user.

Thanks!

Sharing Databricks Community Profile & Achievements on Social Media

TheOC — Tue, 19 Aug 2025 13:03:40 GMT

Hello,
It would be great to be able to more easily share Community Activity / Achievements on social media.
For example:

When earning a new badge, a small pop-up (or, link in the email) to share my achievement on LinkedIN, X, etc. When clicked, this would redirect to the chosen platform with a pre-populated post: "I just earned the badge:...".
This could even be expanded to milestones like accepted solutions or event participation.

I think this would be a cool way of increasing the reward for hitting milestones, but also increase the exposure Databricks has on social media - as people will naturally start to share more around what they're achieving.

Thanks!

'Fun' Community badges

TheOC — Tue, 19 Aug 2025 12:58:34 GMT

Hi - It would be cool to have more 'fun' badges on the Community to collect. Right now they're great for recognising activity, but I think they could be expanded to generate different types of engagement. Some ideas that come to mind:

Attending a user Group
Leading a User Group
Attending a Databricks conference
Getting certified
Posting in particular threads (e.g. Christmas event, Sharing a tip/trick or use-case etc).

I think these kinds of badges would make the Community feel even more interactive / rewarding. What do you think?

More customisation within User Groups

Pilsner — Tue, 19 Aug 2025 12:23:58 GMT

I think it would be beneficial if User Group leaders had more control/customisation over their User Group. This could take many forms, but some ideas I have include:

1) Only owners or approved members can create a new "Topic". All members should be able to reply to a post, but to help maintain a streamlined and clean appearance on the group's landing page, I think the creation of new topics could be a configurable restriction.

2) I like the options of Open, Closed and Hidden group types; however, more customisation here would be great. Could we allow anyone to view the User Group, but only members to reply?

3) Create dedicated subsections. I can see there is a contents section on the user group home page but the only options are "Forums" and "Event". Can we create our own custom subsections?

I don't believe the above list is perfect by any means but I do believe there could be a lot of value in allowing User Group leaders to create a more custom space, tailored to both their style their members.

Regards - Pilsner

DataBricks Community API

TheOC — Tue, 19 Aug 2025 11:44:55 GMT

Would it be possible to get a Community API for the Databricks Community?
This would make it really cool to do some analysis on Community users, ranks, levels, scan for questions that have not yet got solutions etc.

Website notifications appear blank!?

BS_THE_ANALYST — Fri, 11 Jul 2025 17:11:06 GMT

Could be a quick solution as maybe I have a setting enabled but the notifications appear blank?

Could be me being nosy, but I like to see what they're about before clicking on them. I'd hope they'd display the username and the website section they're responding to.

All the best,
BS

Less aggressive Captcha for verified users

TheOC — Fri, 11 Jul 2025 16:35:30 GMT

Hi there,
I love engaging with the Community - but it's slightly tedious that even after 50+ posts, I still have to fill out a captca each time.

I do agree botting/web scrapers are an issue in 2025, but it'd be great to have the system less aggressive for established users.

Cheers,
TheOC

Community Dark Mode

TheOC — Fri, 11 Jul 2025 16:26:12 GMT

Could we get a dark mode for the Community, to match the platform, please?

Dark mode is so much better, my eyes hurt.