Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
Hello professionals and seasoned functional consultants,I recently completed the Databricks Data Engineer Associate certification and have also finished multiple courses from Udemy, Databricks learning events, and Alex The Analyst / Analytics Builder...
I'm unable to start my SQL warehouse (Serverless) due to a RESOURCE_EXHAUSTED error.Error message: Clusters are failing to launch. Cluster launch will be retried. Request to create a cluster failed with an exception: RESOURCE_EXHAUSTED: Cannot create...
Hi @sharath007, Just checked internally. This specific RESOURCE_EXHAUSTED: Cannot create the resource, please try again later message for a serverless SQL warehouse normally indicates that the backing serverless compute pool has run out of capacity (...
Hello, I was wondering if there was sample code indicating how a java program might leverage Databricks Connect to query the table in the Free Edition of Databricks? I would like to use Connect as I am trying to avoid JDBC and its overhead and t...
Hi — welcome to Databricks! Unfortunately, Databricks Connect v2 (DBR 13.3+) does not support Java — it only supports Python, Scala, and R. The legacy v1 did support Java, but it's been deprecated and is end-of-support. That said, here are your optio...
The documentation gives an example of a period to date measure. The intention is to give the sum (in this example) of orders in the current year to date. However, what actually happens is that the measure returns the result for the most recent period...
Hi Ashwin. Thanks for looking into this. Your description of the behaviour and the options matches my understanding, but from my perspective this is a defect: I don't think there's a reasonable interpretation of "period to date" (which is the documen...
Hi - I saw in this dbx post that there is color scale as a column display type. https://docs.databricks.com/aws/en/dashboards/manage/visualizations/tablesHowever I don't see color scale as an option on my dbx. I only see tables for Basic, Image, HTML...
Hi @brianz, @balajij8 is right. You can see from the docs that for table visualisations, the Display type list is Basic, Image, HTML, JSON (no Colour scale). For pivot tables, under Styles → Display type for fields added as Values, the options inclu...
Hello everyone,I am trying to create a cumulative support ticket chart. I need one line representing the cumulative sum of tickets by creation date, and another line for the cumulative sum by resolution date.While it's easy to achieve this by creatin...
Hi @lucca_luna, Great question. The cross-filtering issue you're hitting is because the dashboard treats separate datasets as independent. The workaround is to reshape your data so both date dimensions (creation and resolution) live in a single colum...
How is Cluster auto-scaling in Databricks different from Dynamic Allocation in Yarn
From spark's documentation dynamic resource can be used on more then just YARN. Is it possible to use it in databricks as well and how does serverless work under the hood in these regards?
I followed the official Databricks documentation("https://docs.databricks.com/en/_extras/notebooks/source/mongodb.html")to integrate MongoDB Atlas with Spark by setting up the MongoDB Spark Connector and configuring the connection string in my Datab...
try using single-user cluster or non-isolation cluster instead of shared cluster
I’m currently working with two workspaces – one for DEV and one for PROD.I’m trying to understand how I can keep the Genie/Dashboard functionalities in sync (mirrored) between these two environments. What is the best way to organize this workflow?Ide...
Hi ATN, Good question. Based on what I know, dataset_catalog and dataset_schema in the dashboard DAB resource currently apply to datasets that use SQL queries (i.e., the queryLines property), but they do not resolve for datasets that reference metr...
When I click on a trace in the Databricks MLflow UI, it defaults to opening a "summary" tab. I have a somewhat complex trace, so this summary is getting bloated and I would like to clean it up.Nowhere, as far as I can tell, is the summary tab describ...
Hi @lkt1, I took a quick look and tested it in my workspace to better understand the issue you mentioned. As of today, the Summary tab isn’t user‑configurable. What’s displayed there and how inputs/outputs are collapsed, or truncated, is enti...
I created an external connection to an Oracle Database 12c Standard Edition Release 12.1.0.2.0.The connection is working, I executed successfully a lot of queries with different filters and joins. But I couldn't find a way to make any date filter pus...
Hi @diogofreitaspe, This is a known behavior with how the Lakehouse Federation Oracle connector serializes date and timestamp literals during predicate pushdown. When Databricks pushes the filter down to Oracle, the date value gets rendered as a bare...
Hi everyone,I’m exploring the new Databricks Metric Views (Semantic Layer) and have two questions regarding programmatic management and UI visualization.1. Parser Disparity: spark.sql vs. SQL WarehouseI'm noticing that CREATE OR REPLACE VIEW ... WITH...
Hi @smpa011, METRIC VIEW DDL ON ALL-PURPOSE COMPUTE (spark.sql / %sql) The CREATE VIEW ... WITH METRICS DDL requires Databricks Runtime 17.2 or above. This applies to both SQL warehouses and all-purpose clusters. If your notebook is attached to a clu...
With the new metric views, I am unable to understand the grouping logic in the following setup:I have a table with timestamps and I define dimensions as follows:dimension: - name: timestamp expr: timestamp - name: date expr: DATE(timestam...
Hi @Malthe, This is a nuanced aspect of how metric views resolve window measure dimensions. The key behavior you are seeing comes down to how the metric view engine matches your query's GROUP BY columns to the dimensions defined in the window clause....
HiWe are in the process of designing and building new silver and gold layers (Star Schema). We will be using Databricks, which is new to the organisation.The silver layer will be modelled using classic 3NF, with SCD 7.The gold layer is a star schema....
Hi @RobTScot, This is a common design decision in lakehouse data modeling, and the right answer depends on the layer, the tooling, and the downstream consumers. Here is a breakdown of the key considerations. SILVER LAYER (3NF WITH SCD 7) For silver t...
Hi everyone!Just wanted to jump on here to see if anyone is having issues with Connecticut not mapping on their choropleth map? We had several we were using to map # of healthcare providers in the country but just noticed that Connecticut was no long...
Hi, This is a known issue that stems from Connecticut's 2022 county reorganization and how the AI/BI Dashboard choropleth map resolves FIPS codes. WHAT'S HAPPENING In June 2022, the US Census Bureau officially replaced Connecticut's 8 traditional cou...
