Databricks Platform Discussions
Dive into comprehensive discussions covering various aspects of the Databricks platform. Join the co...
16686 Posts
Activity in Discussions
- 26 Views
- 1 replies
- 0 kudos
X_NHC_CONTROL_PLANE_UNREACHABLE
I have this issue[details] X_NHC_CONTROL_PLANE_UNREACHABLE: Instance failed network health check before bootstrapping with fatal error: X_NHC_CONTROL_PLANE_UNREACHABLE 2 failed component(s): control_plane internet Retryable: true Based on the failure...
- 26 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @taglud_dbw ,The first thing I would check: does the subnet actually have outbound Internet access?This is especially important for a recently created VNet/workspace. Azure changed its default behavior: for new VNets, subnets now default to being ...
- 0 kudos
- 31 Views
- 1 replies
- 0 kudos
Skipping malformed records when reading Avro-files
Using read_files to read Avro-formatted blobs from an external location, I get the error:> Malformed records are detected in record parsing: Event 0. Parse Mode: FAILFAST. To process malformed records as null result, try setting the option 'mode' as ...
- 31 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Malthe ,read_files accepts format-specific options, but the Avro option table has an important scope column. For mode, that scope is only from_avro, not Avro file reads. Spark documents it as:"mode … allows you to specify parse mode for function ...
- 0 kudos
- 57 Views
- 5 replies
- 2 kudos
Databricks streamlit app with write back capability and audit trail display
Hi Everyone, This is a follow up to a previous question I posted. We have a need for a number of apps that have a spreadsheet-like interface and read/write to our lakehouse. A common spec that comes up in these discussions is the need for managers to...
- 57 Views
- 5 replies
- 2 kudos
Latest Reply
- 2 kudos
Thank you both for those very quick and excellent responses. The hovering thing is just an example of the kind of question I get, probably not an exact spec. I would be happy if the audit trail can be displayed somehow in the app.
- 2 kudos
- 106 Views
- 4 replies
- 1 kudos
Resolved! Unity Catalog Metric Views to be accessible to Custom Apps outside of DBX environment
Hello Everyone,We have a requirement to expose Databricks Unity Catalog Metric Views to custom applications running outside the Databricks environment.The application team would like to consume governed KPIs and metrics through APIs such as REST or G...
- 106 Views
- 4 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @vkondepati ,Hi ,Hi,1. Out-of-the-box REST/GraphQL for metric views?Databricks does not appear to provide a Metric View-specific REST or GraphQL query API. Metric Views are queried through the Databricks SQL engine using SQL, including the MEASUR...
- 1 kudos
- 89 Views
- 2 replies
- 2 kudos
Resolved! Data Drift Metrics
As part of flawless data governance and data quality implementation, Data Drift metrics play a very important role. I have compiled a few Data Drift metrics like Jensen-Shannon Divergence Score, Correlation Breakdown Detection, Data Lineage Health an...
- 89 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi Surya,Statistical Drift Metrics to handle distribution shifts across numeric and categorical featuresPopulation Stability Index - Excellent for tracking numeric feature distribution shifts and detecting gradual concept drift over time.Kolmogorov-S...
- 2 kudos
- 405 Views
- 3 replies
- 2 kudos
Resolved! Has anyone prototyped Databricks Lakebase or deployed it in a production environment?
Has anyone prototyped Databricks Lakebase or deployed it in a production environment? I'd be interested in hearing about your experience, including any lessons learned, performance observations, or challenges you encountered.I’ve spent quite a bit of...
- 405 Views
- 3 replies
- 2 kudos
Latest Reply
- 2 kudos
-->Every enterprise runs two database worlds: Postgres/Oracle for transactions, and a Lakehouse for analytics traditionally connected by a fragile CDC pipeline. -->Databricks now runs your transactions AND your analytics on one platform, one governed...
- 2 kudos
- 90 Views
- 2 replies
- 0 kudos
How to remove/delete a Relationship Graph (Semantic Model) from an AI/BI Dashboard?
Hi everyone,I'm working on an AI/BI dashboard and I accidentally initialized a Relationship Graph (semantic model) by clicking "Get started with Relationships" in the Data tab. The model has no relationships and no measures -- it just auto-created en...
- 90 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi brickuser,You have hit a UI limitation with the Dashboard Relationships (Semantic Model) feature in Public Preview. Once you click Get started with Relationships, Databricks automatically initializes a semantic model from your existing datasets. W...
- 0 kudos
- 225 Views
- 1 replies
- 0 kudos
Databricks Training - DNS Issue?
We have some users who are unable to access Databricks training by using SSO. They start with Databricks Training Login: Data Engineering & AI Learning | Databricks and then when choosing Continue with Azure login option from they get this error pag...
- 225 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi Andy, Good detective work here. You checked public DNS before posting, and that screenshot tells most of the story. As far as I can tell, login.cloud.databricks.com isn't a published Databricks endpoint. The documented login hosts are login.databr...
- 0 kudos
- 184 Views
- 4 replies
- 0 kudos
NetSuite connector (Lakeflow Connect) — transactionline refresh takes 3.5–4.8 hrs regardless
Setup: Lakeflow Connect Netsuite pipeline (UC Connection, deployed via UI), Azure Databricks. Target table is netsuite.transactionline, currently about 13.8M rows, 395 columns, CDC ingestion type (I know I know but there aren't that many rows changin...
- 184 Views
- 4 replies
- 0 kudos
Latest Reply
- 0 kudos
Hey there, we offer a best-in-class ELT tool that can connect NetSuite to Azure Databricks out of the box.It supports incremental loading.Might you be interested in this tool?https://studio.precog.cloud
- 0 kudos
- 140 Views
- 3 replies
- 1 kudos
Unity Catalog service credential get_token rejects api:// scope format — "not a valid URI"
Calling get_token() on a Unity Catalog service credential fails for any scope using the api:// App ID URI format. Only https://-scheme resource scopes succeed. The same api:// scopes work correctly with a service principal.Reproductioncredential = db...
- 140 Views
- 3 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @Oumeima ,Totally understand. Let's hope support comes through with a fix
- 1 kudos
- 346 Views
- 6 replies
- 16 kudos
Why is the default auto-termination for serverless interactive notebook compute 60 minutes?
We've been digging into serverless interactive compute costs in notebooks, because our internal cost monitoring (cross-referencing system.billing.usage with system.query.history) showed a large share of the spend coming from idle time rather than act...
- 346 Views
- 6 replies
- 16 kudos
Latest Reply
- 16 kudos
Hi @elizeu_reis , great detective work here. Cross-referencing system.billing.usage with system.query.history is exactly the right way to separate real usage from idle burn. I did some digging, and I can confirm most of what you observed. There are t...
- 16 kudos
- 57 Views
- 1 replies
- 0 kudos
Unexpected behavior of Delta VACUUM – need explanation
HelloI have a question about how VACUUM works.We executed the following VACUUM command from a PySpark job:spark.sql("""VACUUM catalog_name.schema_name.table_name RETAIN 168 HOURS""")The Delta table has a table property retention period of 365 days, b...
- 57 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Greetings @bricks_2026 , I did some digging and here is what I found. First, good question, and you've already pulled the right evidence from the table history. This looks like retention semantics rather than a PySpark quoting issue or a logging bu...
- 0 kudos
- 2630 Views
- 6 replies
- 3 kudos
Resolved! GDPR/CCPA Compliance Delete for PII data
Hi,I am currently designing a PII governance framework to meet CCPA compliance requirements on Databricks. I understand that Databricks provides mechanisms such as VACUUM and Deletion Vectors combined with REORG … APPLY (PURGE) to permanently remove ...
- 2630 Views
- 6 replies
- 3 kudos
Latest Reply
- 3 kudos
For GDPR/CCPA deletion requests involving PII, I think the key is having a clearly defined process that identifies all locations where the user’s data may exist, including primary records, backups, logs, integrations, and derived data.It’s also impor...
- 3 kudos
- 60 Views
- 0 replies
- 0 kudos
Databricks App architecture with AppKit - agentic app with governed write-back functionality
Hi Everyone,I am currently working on an internal application for editing and analyzing forecast data built and hosted via Databricks Apps. I would appreciate a sanity check from anyone who has taken AppKit into production as I am planning on utilizi...
- 60 Views
- 0 replies
- 0 kudos
- 1834 Views
- 3 replies
- 4 kudos
Resolved! Unexpected Script Execution Differences on databricks.com vs Mobile-Triggered Runtimes
I’m noticing some unusual inconsistencies in how scripts execute on databricks.com compared to when the same workflow is triggered through a mobile-based API. On Databricks, the script runs perfectly when executed directly inside a cluster notebook. ...
- 1834 Views
- 3 replies
- 4 kudos
Latest Reply
- 4 kudos
The first thing I’d check is whether both executions are actually running under the same conditions. API-triggered jobs can have different initialization timing, environment variables, dependencies, permissions, or cluster state compared with a noteb...
- 4 kudos
| User | Count |
|---|---|
| 1899 | |
| 1038 | |
| 931 | |
| 480 | |
| 372 |
