Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
Hi All,Does anyone noticed if you "run all" the first time on the notebook, later if you click "run all below" on a cell, that wouldn't work anymore, and require to click "run all" again.It doesn't happen to me about couple of weeks ago, I used to ru...
Hello @Paddy_chu! This doesn't happen with every notebook, but it's likely due to cell dependencies. When you Run All, the final cell might modify or clear the data. Later, if you use Run All Below from a middle cell, it may not work if the required ...
Hey all,I have encountered a strange issue while validating my dashboard visual against the source table.The visual shows the count(distinct VAL) per day and when I ran a query that does the same calculation I get a difference of 84 (on average) and ...
Hi @yuvala, Are you using same query for the dashboard? could you please share it?
Hi All,How can I maintain 7 years of transactional data in delta table? Can I have log retention of 7 days, but data retention of 7 years?Appreciate your response.Thanks and regards Suhas
I would like to be able to express something like: WHERE columnA = "valueA" OR columnB = "valueB" but on the Widget. I only see the possibility that chaining filters that are processed as AND operator. Am I missing something?
Hello,I'm not sure if it is the correct place to post this, sorry.Migrating from an on-premise Oracle to Databricks, we are wondering about the following functionality:. From the reporting tool in place (currenly, PowerBI), users are able to send bac...
Hello Mantu,Thanks for your answer.This is clear, and as our colleague on PowerBI is already dealing with Power Automate, he should be able to test this.If we will be allowed to use Databricks REST API (our infra guys will tell us), I guess we will b...
We are unable to create a Serverless Warehouse Cluster at our own databricks workspace. The same settings do work on other Azure tenants that I have access to.The workspace is running in Azure on a Premium Plan in West Europe.Features enabled:Automat...
Was this workspace moved to Premium plan recently or have been Premium since creation?
Hi-We have a delta table in our unity catalog called dream_team.stern_portfolio.location_info.We are trying to use row level security to filter our data based on a users group membership. This way when users look at out dashboard they can only see th...
Hey @HeathDG1 I think your function isn’t behaving the way you expect because of how the logic is set up:•If the user is in Stern MA, they only see rows where state = 'MA' (which is good).•BUT for everyone else, the function returns true, meaning th...
I would like to know if the cluster size of a Serverless Warehouse can automatically scale up and down, and what determines the number of workers used when executing queries. Does it use all workers within the cluster size fixedly, or does it use par...
@fishingrod My understanding is that Intelligent Workload Management (IWM) in Serverless SQL Warehouses adjusts the number of clusters, but it does not automatically scale the cluster size.This means that if you need to improve the execution performa...
I notice it is very easily to get visualization from sql language inside Databricks. Say you run a SQL query which gives you a table, and you can easily use that table to do its visualization in terms of plots. How about in Python language when we ...
Yes! You can visualize a Python DataFrame in Databricks easily using: display(df)This works like SQL visualizations, offering built-in charts. For more customization, Matplotlib, Seaborn, or Plotly can be used. Would love to see even more native sup...
I recently updated my Databricks VS Code extension from version 1.4.0 to 2.4.0. After the update, I noticed that:The "Configure Sync Destination" button is missing.The sync destination format has changed to a remote folder instead of the previous set...
Hi all,I have a table named employee in Databricks. I ran the following query to filter out rows where the salary is greater than 25000.This query returns 10 rows. I want to find the size of these 10 rows in bytes, and I would like to calculate or re...
Hi @Akshay_Petkar, You can try with this query: SELECT SUM(LENGTH(CAST(employee.* AS STRING))) AS total_size_in_bytesFROM employeeWHERE salary > 25000;
I am using a Shared Databricks Compute and trying to read data from an S3 bucket via an Instance Profile. However, I am encountering the following error: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission SELECT on any ...
Hi @vidya_kothavale , Greetings! Can you please refer to this article and check if it helps you to resolve your issue : https://kb.databricks.com/en_US/data/user-does-not-have-permission-select-on-any-file Please note that these permissions are only ...
I have created a Ai/Bi dashboard on Databricks and set it to refresh every 1 minute. However, the dashboard is not refreshing as expected. Interestingly, when I terminate the warehouse, it triggers the warehouse but still does not refresh per minute ...
Hi @Alberto_Umana ,I have followed the correct approach to create and schedule the dashboard for a 1-minute refresh, but it's not updating every minute as expected.I have attached images for your reference. Please take a look.
Hello Let's say I create a SQL Warehouse on Current Channel (2024.40) and there is a new release (2024.50).Would I need to take some actions(reboot for example) so that my warehouse uses the 2024.50 version or it should run on the 2024.50 whenever th...
Hey @onlyme ,The Channel in Databricks SQL Warehouse has two options:1.Current: This corresponds to the latest stable version released by Databricks and updates automatically.2.Preview: Similar to a beta version, it includes improvements and new feat...
Hello everyone,I created a report using PowerBI Desktop that I successfully connected to Databricks. However, in PBI Service, the visuals are not displayed and I'm asked to edit credentials of the semantic model. When doing so, I get the following er...
Hi @GarCab, Can you try with a PAT token, just to confirm it works? https://learn.microsoft.com/en-us/azure/databricks/partners/bi/power-bi#connect-power-bi-desktop-to-azure-databricks
