cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Discussions
Engage in dynamic conversations covering diverse topics within the Databricks Community. Explore discussions on data engineering, machine learning, and more. Join the conversation and expand your knowledge base with insights from experts and peers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Browse the Community

Community Discussions

Engage in vibrant discussions covering diverse learning topics within the Databricks Community. Expl...

4661 Posts

Activity in Discussions

GabeMatch
by > • New Contributor
  • 18 Views
  • 0 replies
  • 0 kudos

Lakeflow connect Native connectors (tik, meta ads, Google Ads) - one table per account

We want to leverage these connectors to pull in marketing spend data. But the docs seem to say that the destination must be unique based on accounts. For Tik, we have a hundred accounts... each account will have a destination table for each object.  ...

Data Engineering
Reply
  • 18 Views
  • 0 replies
  • 0 kudos
Reply
AndyRoyle
by > • New Contributor III
  • 104 Views
  • 7 replies
  • 1 kudos

Resolved! Dashboard sharing settings

When attempting to change sharing settings of published dashboards from Only people with access can view to anyone in my account can view or vice versa we get a "Failed to change permissions on dashboard..." error. This happens for all dashboards, bo...

Administration & Architecture
Reply
  • 104 Views
  • 7 replies
  • 1 kudos
Reply
Latest Reply
Kirankumarbs
Contributor III
  • 1 kudos

Wow @Ashwin_DSA that's a quick way to escalate and resolve!

  • 1 kudos
6 More Replies
AustinBen
by > • Visitor
  • 28 Views
  • 0 replies
  • 0 kudos

Streaming Amazon DocumentDB to Databricks in near real time - what's the best approach?

Hi everyone,I'm looking for advice from anyone who has implemented near real-time ingestion from Amazon DocumentDB into Databricks.Our current architecture is:Application → Amazon DocumentDBPython AWS Lambda functions capture changes from DocumentDBL...

Data Engineering
Reply
  • 28 Views
  • 0 replies
  • 0 kudos
Reply
VikasM
by > • New Contributor
  • 192 Views
  • 13 replies
  • 5 kudos

Resolved! PySpark AnalysisException: Ambiguous reference to field t when parsing nested JSON

I'm working on a personal data engineering project using Kafka, Spark Structured Streaming, and Docker.The application consumes two Kafka topics that originate from an external market-data websocket source:a trade streama candlestick (kline/OHLCV) st...

Data Engineering
Reply
  • 192 Views
  • 13 replies
  • 5 kudos
Reply
Latest Reply
balajij8
Contributor III
  • 5 kudos

Spark Structured Streaming writes to file sinks and generally it uses a phased commit by writing temporary files to the output directory followed by writing metadata with references and a final commit by moving/renaming temp files to final names. You...

  • 5 kudos
12 More Replies
faruk
by > • New Contributor II
  • 98 Views
  • 3 replies
  • 2 kudos

Data Dictionary and Unity Catalog: Best Practices in a Medallion Architecture

Hi everyone,I'm currently working on implementing a data dictionary, and I'm struggling to understand what the best approach is when using a Medallion architecture in Databricks.My company is migrating its data to Databricks so that users can consume...

Data Governance
Reply
  • 98 Views
  • 3 replies
  • 2 kudos
Reply
Latest Reply
emma_s
Databricks Employee
  • 2 kudos

Hi, This is a challenging area to get right, especially with so many tables and schemas. Unity Catalog is definitely a good bet for storing this metadata. There are a few approaches that you could take to the data definitions part, and it depends on ...

  • 2 kudos
2 More Replies
Dietric
by > • New Contributor II
  • 259 Views
  • 2 replies
  • 1 kudos

Resolved! Databricks apps versus ServiceNow apps

I'm a ServiceNow developer evaluating Databricks as an alternative platform for app development. My pain points with ServiceNow include:Limited-to-zero support for developing apps in popular modern frameworks like React, Flask, and so on.Buggy implem...

Get Started Discussions
Reply
  • 259 Views
  • 2 replies
  • 1 kudos
Reply
Latest Reply
Lu_Wang_ENB_DBX
Databricks Employee
  • 1 kudos

TLDR: Databricks Apps is not a 1:1 ServiceNow replacement. Its intended uses are internal web apps tied to data/AI, especially data entry forms and custom operational interfaces; it supports Python or Node.js, React, local IDE-based development, Git-...

  • 1 kudos
1 More Replies
sandyveeru
by > • Visitor
  • 43 Views
  • 0 replies
  • 0 kudos

Genie not able to list MCP prompts

Hi community,I'm building a custom MCP proxy server (hosted as a Databricks App) that forwards requests to an upstream MCP server. The upstream server exposes both **tools** and **prompts** as per the MCP specification.I can confirm that Genie Code s...

Generative AI
Reply
  • 43 Views
  • 0 replies
  • 0 kudos
Reply
bianca_unifeye
by > • Databricks MVP
  • 42 Views
  • 0 replies
  • 1 kudos

From Innovation to Business Impact: Data + AI Summit Executive Debrief

Hi everyone! I hope you're doing well.Many of you know that the Databricks Data + AI Summit is the biggest Data & AI event of the year. This year was particularly special for me, it was my second consecutive year speaking live on stage in San Francis...

ChatGPT Image Jun 26, 2026, 01_33_15 PM.png
Get Started Discussions
Reply
  • 42 Views
  • 0 replies
  • 1 kudos
Reply
humayun
by > • New Contributor
  • 6764 Views
  • 2 replies
  • 1 kudos

Deprecation of DBFS?

I am using Azure Databricks DBFS. I have stored some python scripts, notebooks in folders in DBFS root and have mounted ADLS Gen2 to DBFS. Currently I do not have Unity Catalog enabled and this was setup 3-4 years ago. I came across this article http...

Data Governance
Reply
  • 6764 Views
  • 2 replies
  • 1 kudos
Reply
Latest Reply
Kirankumarbs
Contributor III
  • 1 kudos

Hey @humayun Storing data DBFS provisioned storage is deprecated but not dbfs:/ URI scheme. Let's not get confused!dbfs:/ scheme to access dbfs:/Volumes/${CATALOG}/libs/wheels is supported and not going to be deprecated!Hope this helps and feel free ...

  • 1 kudos
1 More Replies
Rupa0503
by > • New Contributor III
  • 59 Views
  • 2 replies
  • 1 kudos

Implementing Row Level Security using ABAC

I have to implement row level Security to single/multiple tables based on roles and we don't want to create separate copies for users this one how can i implement and what is the code i can use?

Data Engineering
Reply
  • 59 Views
  • 2 replies
  • 1 kudos
Reply
Latest Reply
Louis_Frolio
Databricks Employee
  • 1 kudos

Hi @Rupa0503 , Yes, you can do row-level security across one table or many in Unity Catalog without copying data per role. @balajij8  pointed you in the right architectural direction (ABAC with governed tags, a reusable row-filter function, and centr...

  • 1 kudos
1 More Replies
gaurang033
by > • New Contributor II
  • 1917 Views
  • 3 replies
  • 2 kudos

how to access snapshots in iceberg tables?

I have created an iceberg tables in databricks, and inserted bunch of values in it. how do I list the snapshot and other metadata of the tables. create table raw.landing.emp_ice(id int, name string ) using icebergfollowing doesn't work https://iceber...

Data Engineering
Reply
  • 1917 Views
  • 3 replies
  • 2 kudos
Reply
Latest Reply
Louis_Frolio
Databricks Employee
  • 2 kudos

@gaurang033 , I believe my solution gets you going in the right direction.  Please give it a read and let me know.  Cheers, Louis.

  • 2 kudos
2 More Replies
Félix_banqi
by > • New Contributor
  • 88 Views
  • 3 replies
  • 0 kudos

Is there a way to deactivate genie auto corretion

Genie keeps breaking my code, sometimes making almost impossible to write code.Sometimes it behaves in a normal way, but sometimes it auto correts at every moment, with non wanted code.There is any way to fix it? I know its a bug, but i also dont kno...

Data Engineering
Reply
  • 88 Views
  • 3 replies
  • 0 kudos
Reply
Latest Reply
Ashwin_DSA
Databricks Employee
  • 0 kudos

Hi @Félix_banqi, Sorry you are facing this issue. That definitely doesn’t sound like the intended experience. I would like to understand the issue better to give you a better steer. Is there an example you can share? In the meantime, given that you h...

  • 0 kudos
2 More Replies
Ajay-Pandey
by > • Databricks MVP
  • 89 Views
  • 2 replies
  • 0 kudos

Agentbricks Knowledge Assistant - Facing Issue with Agent Creation and Source Visibility

Hi Team,I am facing an issue while creating a Knowledge Assistant in the Agent section. It was working fine earlier, and I had uploaded multiple PDF files (70K). However, it has now started behaving unexpectedly.I am not able to see any source data i...

AjayPandey_0-1782362411750.png AjayPandey_1-1782362453412.png AjayPandey_2-1782362563886.png
Generative AI
Reply
  • 89 Views
  • 2 replies
  • 0 kudos
Reply
Latest Reply
Ajay-Pandey
Databricks MVP
  • 0 kudos

Thanks @Ashwin_DSA I have already tried the provided solution, but I am still facing the error. I have also raised a support request with Azure Databricks; however, it has been going in circles for the past month without resolution.

  • 0 kudos
1 More Replies
ss6
by > • New Contributor II
  • 95 Views
  • 1 replies
  • 0 kudos

Account does not exists error

I am writing to request assistance with a login issue I am experiencing with Databricks.I am attempting to create a Databricks account on AWS using my email address. Although I already have an active AWS account associated with this email, I am unabl...

ss6_3-1782414727390.png ss6_4-1782414762504.png ss6_5-1782414833572.png ss6_6-1782414887854.png
Administration & Architecture
Reply
  • 95 Views
  • 1 replies
  • 0 kudos
Reply
Latest Reply
emma_s
Databricks Employee
  • 0 kudos

Hi, For this kind of issue you should email onboarding-help@databricks.com with your account details and they should be able to help. You should have received an onboarding email, which you would have needed to accept before the workspace setup. If y...

  • 0 kudos
PumpItUpper
by > • New Contributor II
  • 73 Views
  • 1 replies
  • 0 kudos

Delta Shared catalog use cost allocation

Hi. I have the following scenario on AWS Databricks:I have a catalog that I need to Delta Share to external systems (both other dbks accounts and other non-dbks systems).All tables on this catalog are MANAGED type, with all the files sitting in the s...

Databricks Free Edition Help
Reply
  • 73 Views
  • 1 replies
  • 0 kudos
Reply
Latest Reply
emma_s
Databricks Employee
  • 0 kudos

Hi As you've rightly said the cost of delta sharing is not in the compute but in the egress costs. I would also ignore the GetObject costs as these again will probably be negligible, around $0.0004 per 1000 get object requests.  So the egress charges...

  • 0 kudos
Load more

Featured Posts

View all