Databricks Community

AmitDECopilot

I have been exploring the architecture behind Agent Bricks Knowledge Assistant, particularly the relationship between Instructed Retriever and the newer Instructed-Retriever-1 model.One practical question helped me understand the difference:“The prod...

AmitDECopilot

One interesting way to look at CDF on Materialized Views is the difference between source-level change and business-level change.CDC might tell us that five banking transactions changed. But after those transactions pass through our transformations,...

AmitDECopilot

In many data platforms, the problem is not that we do not have enough tables.Usually, we have too many.Bronze tables.Silver tables.Gold tables.Reporting views.Power BI calculations.Tableau logic.Excel extracts.Ad-hoc SQL.Notebook logic.After some tim...

AmitDECopilot · 07-18-2026

Performance optimization in Databricks used to follow a familiar playbook:Partition large Delta tables.Compact small files with OPTIMIZE.apply ZORDER BY on frequently filtered columns.Run VACUUM.Collect statistics.Increase cluster size when queries r...

AmitDECopilot · 07-07-2026

AI-assisted coding is changing how data engineers build on Databricks.A developer can now generate Spark SQL, PySpark transformations, Delta table DDL, pipeline logic, data-quality checks, documentation, and test scenarios much faster than before. W...

AmitDECopilot

Great question. I’d generally avoid patching the already-computed Gold table directly.For late-arriving or corrected records, I prefer replaying from the earliest reliable layer where the correction exists, but only for the impacted keys/time window ...

AmitDECopilot

Hi,I think this may be more of an account/subscription mapping issue than customer vs partner sign-up only.Few things I would check:Are you logged into the same Databricks Academy account where the Labs subscription is active?Are you going through th...

AmitDECopilot

Hi,I think first thing to check is whether this is expected auto-stop behavior or the app is actually crashing.In Databricks Apps, if the app is Stopped, normally the app config is still there and you should be able to restart it. But if it is Crashe...

AmitDECopilot · 07-19-2026

My understanding is Databricks AI/BI dashboards are not really built like Grafana where you can install custom visualization plug-ins inside the dashboard itself. So if the requirement is true custom 3D charts using ObservableHQ / Three.js / custom J...

AmitDECopilot · 07-19-2026

I think this is mostly expected behavior with Databricks Asset Bundles, not really a bug.In bundle deployment, the local bundle config is usually the source of truth. So when you change databricks.yml or any included YAML locally and run deploy, Data...

Databricks Community

User Stats

User Activity

Exploring Databricks Instructed-Retriever-1 Through a Data Engineering Use Case

Change Data Feed on Materialized Views Why I Think This Is More Than an Incremental Processing

Databricks Metric Views: Why Gold Tables Are No Longer the End of the Pipeline

Databricks Performance Optimization: What Changed, What Still Matters, and What Should Be Automated

AI Technical Debt in Databricks: Why Generated Spark SQL and PySpark Still Need Metadata, Review

Re: Change Data Feed on Materialized Views Why I Think This Is More Than an Incremental Processing

Re: Labs

Re: My Apps get stopped automatically

Re: Building Plug-ins for AI/BI Dashboard

Re: Databricks YAML files