Databricks Community

Hi @Nidhig ,1. Regarding pipeline cost - here you're mostly paying for compute usage. So the exact price depends on which plan you are at and which cloud provider you are using. For instance, for Azure premium plan and US East region you have followi...

Thanks for weighing in. Are you saying CREATE OR REFRESH STREAMING TABLE user.demo.test_bronze_st  cannot be used in FREE Edition?If we can use it, how do to create STREAM read_files('/Volumes/xxx_ws/demo/raw_files/test.csv'),where .csv sitting on lo...

@ShivMukesh I have used UCX to migrate to Unity catalog. It is a great tool. But it also involves lot of workarounds specially in group migration and table migration. In group migration it renames the old workspace group and assigns the same permissi...

Hi @Han_bbb ,Could you provide more details? Is it workspace in your work or your private one? Which cloud provider? 

This issue is related to how Delta Lake’s structured streaming interacts with schema evolution and options like startingVersion and schemaTrackingLocation. The behavior you've observed has been noted by other users, and can be subtle due to how check...

There is indeed a better and more integrated way to make Delta Lake tables accessible in Redshift without manually generating manifests and dynamically creating external tables or partitions. Some important points and options: Databricks Delta Lake ...

The most effective way to display joined data from Azure Databricks tables (like Employee and Salary) in a React JS UI involves exposing your Databricks data through an API and then consuming that API in your frontend. Flask can work, but there are b...

Auto Loader file notification in Databricks relies on Azure Event Grid’s BlobCreated event to trigger notifications for newly created files in Azure Data Lake Gen2. The issue you’re experiencing is a known limitation when files are written via certai...

Your challenge—reliably determining the subset of changed Databricks Asset Bundles after a Merge Request (MR) is merged into main for focused deploy/destroy CI/CD actions—is common in complex monorepo, multi-environment setups. Let’s break down the p...

Current Approach Assessment Power BI Import Mode: Importing all table data results in full dataset refreshes, driving up compute and data transfer costs during each refresh. Delta Lake as Source: Databricks clusters are used for both ETL and respon...