cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Databricks Platform Discussions
Dive into comprehensive discussions covering various aspects of the Databricks platform. Join the conversation to deepen your understanding and maximize your usage of the Databricks platform.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Browse the Community

Data Engineering

Join discussions on data engineering best practices, architectures, and optimization strategies with...

11361 Posts

Data Governance

Join discussions on data governance practices, compliance, and security within the Databricks Commun...

479 Posts

Generative AI

Explore discussions on generative artificial intelligence techniques and applications within the Dat...

250 Posts

Machine Learning

Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithm...

966 Posts

Warehousing & Analytics

Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Communi...

625 Posts

Activity in Databricks Platform Discussions

genebaldorios
by > • New Contributor
  • 195 Views
  • 2 replies
  • 0 kudos

Resolved! custom calculation of percentage of total and cumulative percentage

Is there a way to calculate the percentage of total using the Databricks dashboard's custom calculations and also the cumulative percentage? What function would be equivalent to the DAX All and AllSelect functions, but using Databricks' aggregation f...

Warehousing & Analytics
Reply
  • 195 Views
  • 2 replies
  • 0 kudos
Reply
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 0 kudos

Hi @genebaldorios ,I don't use databricks dashboards on my project (we are PBI shop), but I guess you need to use AGGREGATE OVER clause with cumulative frame: 

  • 0 kudos
1 More Replies
mohnishbhagde
by > • New Contributor III
  • 685 Views
  • 8 replies
  • 1 kudos

PERMISSION_DENIED: Endpoint databricks-claude-3-7-sonnet is not allowed

I am working on a use case in my personal Databricks account - AgenticDataPipeline. I am getting below error when using  Anthropic Claude 3.7 Sonnet AI model15:52:56.267 | ERROR | [Pipeline] Task failed: ACCOUNT DATA NORMALIZATION: ops_bronze.account...

Machine Learning
Reply
  • 685 Views
  • 8 replies
  • 1 kudos
Reply
Latest Reply
sharonj
Visitor
  • 1 kudos

My name is Sharon J, a developer at sour sop farm, fungi, global mounts 

  • 1 kudos
7 More Replies
BMex
by > • New Contributor III
  • 328 Views
  • 3 replies
  • 2 kudos

Resolved! Issue with Databricks Jobs: SQLSTATE: XXKST

Hi,we have our Databricks Jobs deployed via DABs, and they have been running fine for a while now (approximately 1 month since we migrated from ADF). However, since yesterday, we are getting a weird issue while writing. See error below:[STREAM_FAILED...

Data Engineering
Databricks
databricks-sql
jobs
spark
sqlstate
Reply
  • 328 Views
  • 3 replies
  • 2 kudos
Reply
Latest Reply
WiliamRosa
New Contributor III
  • 2 kudos

Hi @BMex,The link I shared with a similar issue contains some solutions — did any of them work for you?

  • 2 kudos
2 More Replies
Bedoonraj
by > • New Contributor
  • 166 Views
  • 2 replies
  • 0 kudos

TEMPORARILY_UNAVAILABLE: The service at /api/2.1/unity-catalog/tables is taking too long to process

I'm using DBT to run a model in databricks. I have view model, which holds 2 months of data (~2 million). There is no wide dependency transformation. All are case when statements. Total column no is 234. Till yesterday view was running fine. but toda...

Data Engineering
Reply
  • 166 Views
  • 2 replies
  • 0 kudos
Reply
Latest Reply
Bedoonraj
New Contributor
  • 0 kudos

I tried for 1-2 days, still same error. I changed to table materialization, then only it worked. Any reason view is failing?

  • 0 kudos
1 More Replies
ManojkMohan
by > • Contributor III
  • 177 Views
  • 1 replies
  • 0 kudos

Databricks ro Salesforce | Unity Catalog Query

AskCan we get a UC catalog (like prod or genie) in free edition of data bricks ?Problem i am solving:Structuring Data in Databricks before sending customer, account data to salesforceissue:cannot see workspace-local tables (workspace.default.structur...

image (1).png image (2).png
Data Engineering
Reply
  • 177 Views
  • 1 replies
  • 0 kudos
Reply
Latest Reply
RogerThatttt
New Contributor II
  • 0 kudos

root cause of not seeing your workspace-local tables (workspace.default.structured_pdf_table) is the unavailability of a Unity Catalog or Delta Sharing connector configuration in your Free Edition workspace. To resolve this, you typically need admin ...

  • 0 kudos
gbhatia
by > • New Contributor II
  • 212 Views
  • 2 replies
  • 1 kudos

Endpoint deployment is very slow

HI team I am testing some changes on UAT / DEV environment and noticed that the model endpoint are very slow to deploy. Since the environment is just testing and not serving any production traffic, I was wondering if there was a way to expedite this ...

Machine Learning
Reply
  • 212 Views
  • 2 replies
  • 1 kudos
Reply
Latest Reply
WiliamRosa
New Contributor III
  • 1 kudos

Hi @gbhatia,I’d need a few more details to fully understand your deployment, but in general, what can help is setting Compute type: CPU (cheaper and sufficient for testing), Compute scale-out: Small (0–4 concurrency, 0–4 DBU) since you don’t need hig...

  • 1 kudos
1 More Replies
shemasri
by > • New Contributor
  • 91 Views
  • 0 replies
  • 0 kudos

Subscribepage

https://subscribepage.io/rhinogoldgel

Warehousing & Analytics
Reply
  • 91 Views
  • 0 replies
  • 0 kudos
Reply
shanisolomonron
by > • New Contributor
  • 176 Views
  • 1 replies
  • 0 kudos

Merge, Update and deletes using deletion vectors

Hi,When deletion vectors are enabled on a Delta table, is there a guarantee that MERGE, UPDATE, or DELETE operations will not rewrite unmodified data, but rather use deletion vectors to soft delete the original file?For example, suppose the table cur...

Data Engineering
Reply
  • 176 Views
  • 1 replies
  • 0 kudos
Reply
Latest Reply
K_Anudeep
Databricks Employee
  • 0 kudos

Hey @shanisolomonron , Yes, you are right. The above sequence of actions is always true for MERGE and UPDATE .For DELETE , you don't see any Add a new file (step 3) And yes, if the table has the DV feature enabled, the writer/runtime supports DVs for...

  • 0 kudos
alxsbn
by > • Contributor
  • 893 Views
  • 1 replies
  • 1 kudos

Databricks Managed MCP server x Claude Code

Hi everyone,I'm looking to connect the managed MCP server to Claude code on Linux and struggle to have something working. The MCP are activated on my workspace (not sure it's linked to what I want to do) and I just want to add the UC endpoint to Clau...

Generative AI
Reply
  • 893 Views
  • 1 replies
  • 1 kudos
Reply
Latest Reply
jamesl
Databricks Employee
  • 1 kudos

Hi @alxsbn , did you get this resolved? Targets that are a remote HTTP endpoint need `--transport http`. For example... claude mcp add --transport http databricks-server https://xx-yyy-zzz.cloud.databricks.com/api/2.0/mcp/gold/core (see: https://docs...

  • 1 kudos
sfishel18
by > • New Contributor II
  • 332 Views
  • 4 replies
  • 0 kudos

GEOMETRY column type breaks all access to table from Spark

Hello, I have a Databricks table with a column using the new GEOMETRY type. When I try to access this table from a Spark workload, I am not able to describe the table or operate on any of its columns. My Spark config is the following, per the Databri...

Data Engineering
Reply
  • 332 Views
  • 4 replies
  • 0 kudos
Reply
Latest Reply
NandiniN
Databricks Employee
  • 0 kudos

Reading the UC OSS doc - https://docs.unitycatalog.io/usage/tables/formats/ it does not support all data types columnsThe columns of the table in SQL-like format "column_name column_data_type". Supported data types include BOOLEAN, BYTE, SHORT, INT, ...

  • 0 kudos
3 More Replies
NUKSY
by > • New Contributor II
  • 981 Views
  • 4 replies
  • 0 kudos

`io.unitycatalog.client.model.TableType`, Unexpected value 'MATERIALIZED_VIEW

I have been able to set up jdbc driver with databricks to connect to my unity catalog using local spark sessions. When i try to retrieve tables in my schema i get this error  An error occurred while calling o43.sql.: io.unitycatalog.client.ApiExcepti...

Data Engineering
Reply
  • 981 Views
  • 4 replies
  • 0 kudos
Reply
Latest Reply
NandiniN
Databricks Employee
  • 0 kudos

Hi @NUKSY , @Jofes  This should be reported as a bug, see similar issues reported. https://github.com/unitycatalog/unitycatalog/issues/657 https://github.com/unitycatalog/unitycatalog/issues/1077 Thanks!

  • 0 kudos
3 More Replies
tenzinpro
by > • New Contributor II
  • 405 Views
  • 2 replies
  • 1 kudos

delta live tables

Hi . i have a source table that is a delta live streaming table created using dlt.auto_cdc logic  and now i want to create another sreaming table that filters the record from that table as per the client but it also should have auto cdc logic for the...

Data Engineering
Reply
  • 405 Views
  • 2 replies
  • 1 kudos
Reply
Latest Reply
NandiniN
Databricks Employee
  • 1 kudos

Hi @tenzinpro ,  This is an expected error. "DELTA_SOURCE_TABLE_IGNORE_CHANGES] Detected a data update" As explained in the error: This is currently not supported. If this is going to happen regularly and you are okay to skip changes, set the option ...

  • 1 kudos
1 More Replies
Michał
by > • New Contributor II
  • 360 Views
  • 3 replies
  • 2 kudos

how to process a streaming lakeflow declarative pipeline in batches

Hi, I've got a problem and I have run out of ideas as to what else I can try. Maybe you can help? I've got a delta table with hundreds millions of records on which I have to perform relatively expensive operations. I'd like to be able to process some...

Data Engineering
Reply
  • 360 Views
  • 3 replies
  • 2 kudos
Reply
Latest Reply
mmayorga
Databricks Employee
  • 2 kudos

Hi @MichaÅ‚ , One detail/feature to consider when working with Declarative Pipelines is that they manage and auto-tune configuration aspects, including rate limiting (maxBytesPerTrigger or maxFilesPerTrigger). Perhaps that's why you could not see this...

  • 2 kudos
2 More Replies
sathyavikasini
by > • New Contributor
  • 404 Views
  • 1 replies
  • 0 kudos

Error while creating Agents with Agentbricks UI

While creating a Knowledge Assistant in Databricks, I encountered an issue where the endpoint update failed with the following message:"Failed to deploy: served entity creation aborted because the endpoint update timed out. Please see service logs fo...

sathyavikasini_0-1754497230496.png sathyavikasini_3-1754497718786.png sathyavikasini_4-1754497745047.png
Generative AI
Reply
  • 404 Views
  • 1 replies
  • 0 kudos
Reply
Latest Reply
jamesl
Databricks Employee
  • 0 kudos

  Hi @sathyavikasini , Are you still facing this issue? While creating the agent, do you see that your configured knowledge sources are processing? If you haven't done so already, you might check permissions on the catalog/schema/volume(s) that are ...

  • 0 kudos
Wasubabu
by > • New Contributor II
  • 294 Views
  • 3 replies
  • 0 kudos

Embed AI/BI Dashboards into Databricks App

Hello,I’m interested in understanding whether it’s possible to embed multiple AI/BI dashboards created in Databricks within a Databricks app. Could you please share the steps or provide any documentation related to this? My goal is to use the app as ...

Data Engineering
Reply
  • 294 Views
  • 3 replies
  • 0 kudos
Reply
Latest Reply
Wasubabu
New Contributor II
  • 0 kudos

Just checking if anyone has already implemented this. Pls share your thoughts.

  • 0 kudos
2 More Replies
Load more

Featured Posts

View all