Databricks Community

 The Challenge Enterprises often have hundreds or even thousands of legacy SQL queries or stored procedures written in platforms like SQL Server, Teradata, or Snowflake.Migrating these to Databricks manually can be slow, tedious, and error-prone — e...

In this post, we’ll explore the two main approaches to integrating UC with other Iceberg environments — Foreign Iceberg tables and Managed Iceberg tables — and walk through when to choose each, how to configure them, and the operational consideration...

Databricks Auto Loader is a powerful tool for efficiently ingesting files from cloud storage. In this blog, we'll focus on setting up Auto Loader in file notification mode on AWS using service credentials. Why Use File Notification Mode? File notific...

As organizations continue to scale their data infrastructure, efficient resource utilization, cost control, and operational transparency are paramount for success. With the growing adoption of Databricks, monitoring and optimizing compute usage and d...

IntroductionCreate S3 bucket and cross-account instance profileUse Autoloader to create SNS-SQS across accountsManually create SNS-SQS for cross-account AutoloaderTest cross-account Autoloader connection Introduction This is a step-by-step guide to s...