Databricks Community is the official forum for data engineers, data scientists, data enthusiasts and AI practitioners to ask questions, share knowledge, and collaborate on everything related to Databricks. Members can find answers, share best practices, and support each other in an environment that ensures respectful interactions.
I have a table pulled from Snowflake via a foreign catalog that's only returning 100 rows in Databricks even though there's billions of records in the source Snowflake table. What steps should I take...
Hi everyone, I'm currently working on implementing a data dictionary, and I'm struggling to understand what the best approach is when using a Medallion architecture in Databricks. My company is mig...
In the article Manage budgets and cost controls for Genie at Manage budgets and cost controls for Genie - Azure Databricks | Microsoft Learn it mentions setting the resource type of Un...
I am setting up a new workspace that will use the Unity Catalog. I want all data stored in the Unity Catalog in the following catalogs: dev, staging, prod. I want to prevent users from accidentally r...
I have all the necessary permissions like USE CATALOG on Catalog, USE SCHEMA on Schema and SELECT,TABLE on Table. But still when I try to access the table from catalog I get the error : Insufficient ...
I changed the administrative permissions for a specific service principal in the account management screen. I expected an audit log to be generated with service_name = accountns and action_name = ch...
The company that I work for has DDL where the column names/table names are not always that intuitive. It would be great to see the "Title" clause such that is used in ANSI SQL, added to the DDL state...
Hello Databricks Community, Databricks is making great progress towards becoming a truly governed and unified catalog. When it comes to governing and cataloguing physical data assets, it already off...
Hello Looking for guidance to keep data masked if it needs to link with unmasked data, or analysis requires a combination of masked and unmasked data. How have you tackle this challenge? Thank you....