- 1171 Views
- 5 replies
- 1 kudos
Databricks Apps - "App Not Available" error with locationId parameter missing
Hello,I'm experiencing an issue with Databricks Apps where I cannot access my deployed app from a browser, even though the app appears to be running successfully.**Symptoms:**- Browser shows "App Not Available" error when accessing the app URL- Conso...
- 1171 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi @yutmurakami — I can now answer your question with data instead of hypotheses. I ran your exact scenario on my own Free Edition workspace (also AWS, different workspace than yours), twice.Test 1: existing app on my Free Edition workspace — Running...
- 1 kudos
- 2377 Views
- 2 replies
- 0 kudos
Databricks "Preferred" Approaches To Backfilling Single Column In Wide Tables
Hi there,I've tried thinking through this and googling as well, but I'm not sure if there's a better approach that I might be missing. We have *wide* tables with hundreds of columns, and on a day-to-day basis these tables are incrementally filled in ...
- 2377 Views
- 2 replies
- 0 kudos
- 0 kudos
We hit exactly this, offline feature store on Redshift, ~200-column denormalized table fed by domain models. Adding a feature meant NULLs for history, and full recompute was too expensive, so we landed on: recompute the domain for the backfill range ...
- 0 kudos
- 504 Views
- 3 replies
- 0 kudos
What happens to with SCD2 tables if the source pipeline performs a Full Refresh?
Hi everyone,I'm looking for clarification on how Lakeflow/DLT behaves in the following scenario.We have two independent pipelines:Pipeline 1 (Ingestion)Ingests Salesforce objects into Delta tables.These tables represent the current snapshot of Salesf...
- 504 Views
- 3 replies
- 0 kudos
- 0 kudos
The pattern is documented and you are right not to full refresh pipeline 2, but as written the deletion detection can never fire, and the full refresh is not why. spark.readStream.table("raw.contact") makes contact append-only, so no key ever leaves ...
- 0 kudos
- 566 Views
- 3 replies
- 0 kudos
Getting start with Data Brick machine creation.
I am a beginner for Data Bricks. Want to learn from the scratch.
- 566 Views
- 3 replies
- 0 kudos
- 0 kudos
Great topic! Databricks machine creation and setup can seem complex at first, but once you understand clusters, compute resources, and workspace configuration, it becomes much easier. A step-by-step guide with screenshots and best practices for begin...
- 0 kudos
- 692 Views
- 2 replies
- 1 kudos
Streaming tables vs materialized views in declarative pipelines: what actually decides the choice?
I am trying to settle a design default for our declarative pipelines, and I would like to hear how others decide between streaming tables and materialized views when a table could be built either way.When I started with streaming tables, I thought th...
- 692 Views
- 2 replies
- 1 kudos
- 1 kudos
Good points, Brahmareddy — and I think we mostly agree, with one nuance worth teasing out on the cross-team part.A published streaming table already *is* a stable Delta table from the reader's perspective — consumers query it by its UC name and never...
- 1 kudos
- 7613 Views
- 4 replies
- 6 kudos
Resolved! Has Unity Catalog been available in Azure Gov Cloud?
We are using Databricks with Premium Tier in Azure Gov Cloud. We check the Data section but don't see any options to Create Metastore.
- 7613 Views
- 4 replies
- 6 kudos
- 871 Views
- 2 replies
- 2 kudos
Resolved! Context engineering for Genie
Hello Team,I have a client who is already using Genie One and Genie Agents, and I want to help them build a strong context layer to improve the quality and accuracy of responses.I'm looking for guidance on the overall approach and best practices for ...
- 871 Views
- 2 replies
- 2 kudos
- 531 Views
- 3 replies
- 2 kudos
CREATE CONNECTION - Support for Community Connections?
Hi there, as the title asks, I'm wondering how Databricks Community Connections are being supported. For example, when I try to make a fresh connection via Spark SQL commands, I get an error because `COMMUNITY` type is not supported.However, when I c...
- 531 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @ChristianRRL I tried with FHIR and it worked. The process is the same for every source. To do GitHub instead of the FHIR example, you change exactly three things, the connector name (github), the credential (a GitHub token), and the list of tabl...
- 2 kudos
- 468 Views
- 2 replies
- 0 kudos
Collation Change Planning & Fixing
We recently decided to change from the default collation of UTF8_BINARY to UTF8_LCASE and there are many issues. We attempted to cascade the collation defaults to all the Catalogs, Schemas, Tables, Views and Columns, but we still seem to have expres...
- 468 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi skipowder99,When you set a DEFAULT COLLATION at the catalog or schema level, it is forward looking. It does not retroactively apply to existing tables, columns or views. It leaves you with a split environment the newly created objects default to U...
- 0 kudos
- 335 Views
- 0 replies
- 1 kudos
V.E.N.K.A.T Framework™
Hello Databricks Community,I recently published the V.E.N.K.A.T Framework™, an enterprise architecture model for the Agentic AI era.The idea is simple:As AI moves from generating insights to taking actions, the architecture underneath becomes critica...
- 335 Views
- 0 replies
- 1 kudos
- 29804 Views
- 16 replies
- 2 kudos
Install the ODBC Driver 17 for SQL Server
Hi,I am not a Data Engineer, I want to connect to ssas. It looks like it can be connected through pyodbc. however looks like I need to install "ODBC Driver 17 for SQL Server" using the following command. How do i install the driver on the cluster an...
- 29804 Views
- 16 replies
- 2 kudos
- 2 kudos
If you're only connecting to SSAS, installing the ODBC Driver 17 for SQL Server is usually the first step, but the exact method depends on your cluster environment and operating system. If it's a managed cluster, you may need admin access or an scrip...
- 2 kudos
- 809 Views
- 2 replies
- 1 kudos
Resolved! Folder structure for Autoloader and Declarative Pipelines
We have six years of Parquet files organised under yyyy/mm/dd/, with around 100 table files in each daily folder. We want to process these 100 tables using Auto Loader in a Declarative Pipeline with the AUTO CDC API.If each table reads the same root ...
- 809 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @batch_bender ,Yes, pointing Auto Loader to the root path with a filename wildcard for every table will cause significant file discovery overhead, as each stream independently crawls the entire directory tree (yyyy/mm/dd/).Best Practice Recommenda...
- 1 kudos
- 397 Views
- 1 replies
- 0 kudos
oreign Iceberg tables via HMS federation fail with browse_only_table on all compute — request Public
Per your documentation (hms-federation-concepts), Apache Iceberg tables in federated external Hive metastores are Public Preview on Databricks Runtime 16.2+, with no enablement steps listed. On our workspace, every read of a federated Iceberg table f...
- 397 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi matko,The underlying issue here is mostly a missing backend feature enablement for the workspace. Databricks frequently gates these newer federation capabilities behind workspace-level or account-level enablement during the preview phase.Unity Cat...
- 0 kudos
- 553 Views
- 1 replies
- 0 kudos
Resolved! Multi-Engine Lakehouses
Hi experts,I recently came across an interesting discussion titled “Are enterprises moving from ‘Data Lakehouse’ to ‘Agentic Lakehouse’?” https://community.databricks.com/t5/data-engineering/are-enterprises-moving-from-quot-data-lakehouse-quot-to-quo...
- 553 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Sam500,The multi engine pattern usually comes to establishing interoperability at the storage format level while maintaining a single, unified governance layer. You handle this primarily through Unity Catalog (UC) and format flexibility such as De...
- 0 kudos
- 503 Views
- 2 replies
- 2 kudos
DLT driver GC pressure during a large initial hydration. Is a bigger driver the only lever?
Last week one of our DLT pipelines spent five hours in RUNNING with nothing landing, and I'd like to compare notes on how people handle the underlying problem, because we found one fix and several dead ends.Setup: A continuous DLT pipeline on classic...
- 503 Views
- 2 replies
- 2 kudos
- 2 kudos
I ran into something similar, and the event log was the only place that made the root cause obvious. I don't believe there's an officially supported setting to limit concurrent DLT flows, so I'd avoid undocumented configs. For large initial backfills...
- 2 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 584 | |
| 349 | |
| 287 |
