Hey everyone,
I'm working on a project to enhance our data analysts' experience on the platform. In our company, we've done a great job with Unity Catalog, and we have comprehensive descriptions for all the tables and columns in our data lake.
I'm interested in creating a chatbot assistant to help our data analysts navigate and understand the data more efficiently. The idea is to have an assistant that knows the entire data dictionary and can answer queries about the data without requiring analysts to manually explore documentation or metadata.
I've tried using the Genie Space feature, but from what I understand, it requires selecting a few tables at a time. I'm looking for a solution where the assistant has access to the entire Unity Catalog metadata, providing a more seamless and holistic experience for the analysts.
Is there any out-of-the-box solution from Databricks that could support this? Or has anyone implemented something similar and can share insights on how to approach this? Any advice or suggestions would be greatly appreciated!
Thanks in advance for your help!