Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
I would like to be able to express something like: WHERE columnA = "valueA" OR columnB = "valueB" but on the Widget. I only see the possibility that chaining filters that are processed as AND operator. Am I missing something?
Hello,I'm not sure if it is the correct place to post this, sorry.Migrating from an on-premise Oracle to Databricks, we are wondering about the following functionality:. From the reporting tool in place (currenly, PowerBI), users are able to send bac...
Hello Mantu,Thanks for your answer.This is clear, and as our colleague on PowerBI is already dealing with Power Automate, he should be able to test this.If we will be allowed to use Databricks REST API (our infra guys will tell us), I guess we will b...
We are unable to create a Serverless Warehouse Cluster at our own databricks workspace. The same settings do work on other Azure tenants that I have access to.The workspace is running in Azure on a Premium Plan in West Europe.Features enabled:Automat...
Was this workspace moved to Premium plan recently or have been Premium since creation?
Hi-We have a delta table in our unity catalog called dream_team.stern_portfolio.location_info.We are trying to use row level security to filter our data based on a users group membership. This way when users look at out dashboard they can only see th...
Hey @HeathDG1 I think your function isn’t behaving the way you expect because of how the logic is set up:•If the user is in Stern MA, they only see rows where state = 'MA' (which is good).•BUT for everyone else, the function returns true, meaning th...
I would like to know if the cluster size of a Serverless Warehouse can automatically scale up and down, and what determines the number of workers used when executing queries. Does it use all workers within the cluster size fixedly, or does it use par...
@fishingrod My understanding is that Intelligent Workload Management (IWM) in Serverless SQL Warehouses adjusts the number of clusters, but it does not automatically scale the cluster size.This means that if you need to improve the execution performa...
I notice it is very easily to get visualization from sql language inside Databricks. Say you run a SQL query which gives you a table, and you can easily use that table to do its visualization in terms of plots. How about in Python language when we ...
Yes! You can visualize a Python DataFrame in Databricks easily using: display(df)This works like SQL visualizations, offering built-in charts. For more customization, Matplotlib, Seaborn, or Plotly can be used. Would love to see even more native sup...
I recently updated my Databricks VS Code extension from version 1.4.0 to 2.4.0. After the update, I noticed that:The "Configure Sync Destination" button is missing.The sync destination format has changed to a remote folder instead of the previous set...
Hi all,I have a table named employee in Databricks. I ran the following query to filter out rows where the salary is greater than 25000.This query returns 10 rows. I want to find the size of these 10 rows in bytes, and I would like to calculate or re...
Hi @Akshay_Petkar, You can try with this query: SELECT SUM(LENGTH(CAST(employee.* AS STRING))) AS total_size_in_bytesFROM employeeWHERE salary > 25000;
I am using a Shared Databricks Compute and trying to read data from an S3 bucket via an Instance Profile. However, I am encountering the following error: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission SELECT on any ...
Hi @vidya_kothavale , Greetings! Can you please refer to this article and check if it helps you to resolve your issue : https://kb.databricks.com/en_US/data/user-does-not-have-permission-select-on-any-file Please note that these permissions are only ...
I have created a Ai/Bi dashboard on Databricks and set it to refresh every 1 minute. However, the dashboard is not refreshing as expected. Interestingly, when I terminate the warehouse, it triggers the warehouse but still does not refresh per minute ...
Hi @Alberto_Umana ,I have followed the correct approach to create and schedule the dashboard for a 1-minute refresh, but it's not updating every minute as expected.I have attached images for your reference. Please take a look.
Hello Let's say I create a SQL Warehouse on Current Channel (2024.40) and there is a new release (2024.50).Would I need to take some actions(reboot for example) so that my warehouse uses the 2024.50 version or it should run on the 2024.50 whenever th...
Hey @onlyme ,The Channel in Databricks SQL Warehouse has two options:1.Current: This corresponds to the latest stable version released by Databricks and updates automatically.2.Preview: Similar to a beta version, it includes improvements and new feat...
Hello everyone,I created a report using PowerBI Desktop that I successfully connected to Databricks. However, in PBI Service, the visuals are not displayed and I'm asked to edit credentials of the semantic model. When doing so, I get the following er...
Hi @GarCab, Can you try with a PAT token, just to confirm it works? https://learn.microsoft.com/en-us/azure/databricks/partners/bi/power-bi#connect-power-bi-desktop-to-azure-databricks
Hi,I think I have a similar issue to the one in this post, but the answer isn't detailed enough for me.I have a list defined in my first task, which contains the items I want to iterate through [1,2,3,4]. When I use it as Inputs to the For Each frami...
Thank you both very much, I've nailed it . I have accepted Walter_C's answer as solution because Step 2 is what I was missing. Thanks MariuszK as well for your contribution.
Do we have same functionality if we use Databricks with Fabric as it provides with Azure?
hi @DEShoaib Are you planing to move Dedicated (T-SQL) pool or Spark code?With Databricks you can replicated all features from Azure Synapse, you have possibility to use PySpark and Databricks SQL. MS Fabric has nice integration with Power BI and eas...
For Databricks SQL connector for python, the list of fields returned by Cursor.columns() is listed in here (like TABLE_CAT, TABLE_SCHEM, TABLE_NAME, COLUMN_NAME). Could someone please share an exhaustive list of fields (including short description ...
All those fields are explained in the doc. For example search with `TABLE_CAT`
