Databricks Community

ShamenParis

Most public GitHub repositories for Databricks are just dry code dumps. You get a massive YAML file, a basic README, and are expected to just figure it out.When building production-grade infrastructure, engineers need to see the architecture and unde...

ShamenParis

Vector search is everywhere right now, but is it always the right tool for the job? Actually, noIf you need to find a specific Trace ID or error code buried in 100 million log rows, you don't want an AI model guessing what's "conceptually similar." Y...

ShamenParis · 06-06-2026

Hi all,Tired of paying the data movement tax or wrestling with complex manual pipeline configs?I just published a new Medium article and open-sourced a framework that fully automates Databricks Lakeflow Connect pipelines for CDC-enabled databases usi...

ShamenParis · 05-31-2026

If you are building enterprise Generative AI, you know the pain of the "Monolithic Agent Bottleneck": passing dozens of tools to a single AI supervisor leads to hallucinated routing, massive context bloat, and security nightmares.With the recent May ...

ShamenParis · 05-31-2026

While AI and LLMs take the headlines, hardcore data engineers know that SQL remains the operational backbone of enterprise pipelines. Databricks just rolled out several powerful programmatic and geospatial updates that solve real-world, complex data ...

ShamenParis

Hi @Aparajita_Nag The "could not be found" error indicates Databricks cannot locate the Access Connector resource in Azure. This happens before any ADLS permissions are checked. Because I lack full information on your setup, you can verify the below ...

ShamenParis

Hi @faruk I understand the situation, but you can show this reply to your security team and try to get a view created. Once you have that access, you can use a recursive function to get the lineage of lineage in memory.Here is the exact logic I used ...

ShamenParis

Hi @faruk ,I completely understand the frustration. Navigating enterprise permissions is a headache, and I actually went down this exact same rabbit hole recently when I was trying to pull lineage data strictly using the Databricks REST API or SDK.To...

ShamenParis

Hi @Oumeima ,Totally understand. Let's hope support comes through with a fix爛

ShamenParis

Hi @Shubh5073 ,As per my understanding of the issue you are facing, the error indicates that Databricks cannot locate the Access Connector resource in Azure, rather than a problem with your ADLS permissions (since you already correctly assigned Stora...

Databricks Community

User Stats

User Activity

Visual, step-by-step curriculum for mastering DABs and Infrastructure as Code

Demystifying Databricks Full Text Search: It’s Not Vector Search (And That’s Okay!)

Automating Databricks Lakeflow Connect Pipelines for CDC Databases

Stop giving your Databricks AI Agents 50 tools to manage

Databricks SQL Just Dropped Some Massive Engine Upgrades for Data Engineers

Re: Unable to Enable Unity Catalog – Azure Managed Identity Credential Not Found

Re: Unity Catalog Lineage APIs – Are these REST APIs publicly supported?

Re: Unity Catalog Lineage APIs – Are these REST APIs publicly supported?

Re: Unity Catalog service credential get_token rejects api:// scope format — "not a valid URI&q

Re: Unable to Enable Unity Catalog – Azure Managed Identity Credential Not Found