Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
Hi-We have a delta table in our unity catalog called dream_team.stern_portfolio.location_info.We are trying to use row level security to filter our data based on a users group membership. This way when users look at out dashboard they can only see th...
Hey @HeathDG1 I think your function isn’t behaving the way you expect because of how the logic is set up:•If the user is in Stern MA, they only see rows where state = 'MA' (which is good).•BUT for everyone else, the function returns true, meaning th...
I would like to know if the cluster size of a Serverless Warehouse can automatically scale up and down, and what determines the number of workers used when executing queries. Does it use all workers within the cluster size fixedly, or does it use par...
@fishingrod My understanding is that Intelligent Workload Management (IWM) in Serverless SQL Warehouses adjusts the number of clusters, but it does not automatically scale the cluster size.This means that if you need to improve the execution performa...
We are launching our first Databricks Apps based on Streamlit.The App works when simply running the notebook in our workspace, but fails after deployment due to "could not find a valid JAVA_HOME installation" when running in the system environment.We...
I notice it is very easily to get visualization from sql language inside Databricks. Say you run a SQL query which gives you a table, and you can easily use that table to do its visualization in terms of plots.How about in Python language when we hav...
Yes! You can visualize a Python DataFrame in Databricks easily using: display(df)This works like SQL visualizations, offering built-in charts. For more customization, Matplotlib, Seaborn, or Plotly can be used. Would love to see even more native sup...
I recently updated my Databricks VS Code extension from version 1.4.0 to 2.4.0. After the update, I noticed that:The "Configure Sync Destination" button is missing.The sync destination format has changed to a remote folder instead of the previous set...
Hi Community!I have the requirement that business users shall be able to edit/update tables in Unity Catalog, e.g. master data records, mapping tables. I also want thes actions to be logged for auditing/troubleshooting.Is there any simple solution to...
We do have a few partners that offer solutions in this space (e.g. Retool). Recently, Sigma added support for their InputTable feature which was designed for this use case: https://www.sigmacomputing.com/blog/bring-your-own-data-to-databricks-with-s...
In a Databricks AI/BI dashboard, I have a field with multiple categories (e.g., district-wise sales with 50 districts). How can I display only the top few categories (like the top 10) based on a specific metric such as sales?
hey @migq2 , @maks in the AI/BI dashboards in your data, add a limit parameter like:select all from my_table limit :limit_number to all your tables. when you're on canvas and adding visualizations, add a filter and create a parameter with single val...
Hi all,I have a table named employee in Databricks. I ran the following query to filter out rows where the salary is greater than 25000.This query returns 10 rows. I want to find the size of these 10 rows in bytes, and I would like to calculate or re...
Hi @Akshay_Petkar, You can try with this query: SELECT SUM(LENGTH(CAST(employee.* AS STRING))) AS total_size_in_bytesFROM employeeWHERE salary > 25000;
I am using a Shared Databricks Compute and trying to read data from an S3 bucket via an Instance Profile. However, I am encountering the following error: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission SELECT on any ...
Hi @vidya_kothavale , Greetings! Can you please refer to this article and check if it helps you to resolve your issue : https://kb.databricks.com/en_US/data/user-does-not-have-permission-select-on-any-file Please note that these permissions are only ...
I have created a Ai/Bi dashboard on Databricks and set it to refresh every 1 minute. However, the dashboard is not refreshing as expected. Interestingly, when I terminate the warehouse, it triggers the warehouse but still does not refresh per minute ...
Hi @Alberto_Umana ,I have followed the correct approach to create and schedule the dashboard for a 1-minute refresh, but it's not updating every minute as expected.I have attached images for your reference. Please take a look.
Hello Let's say I create a SQL Warehouse on Current Channel (2024.40) and there is a new release (2024.50).Would I need to take some actions(reboot for example) so that my warehouse uses the 2024.50 version or it should run on the 2024.50 whenever th...
Hey @onlyme ,The Channel in Databricks SQL Warehouse has two options:1.Current: This corresponds to the latest stable version released by Databricks and updates automatically.2.Preview: Similar to a beta version, it includes improvements and new feat...
Hello everyone,I created a report using PowerBI Desktop that I successfully connected to Databricks. However, in PBI Service, the visuals are not displayed and I'm asked to edit credentials of the semantic model. When doing so, I get the following er...
Hi @GarCab, Can you try with a PAT token, just to confirm it works? https://learn.microsoft.com/en-us/azure/databricks/partners/bi/power-bi#connect-power-bi-desktop-to-azure-databricks
Hi,I think I have a similar issue to the one in this post, but the answer isn't detailed enough for me.I have a list defined in my first task, which contains the items I want to iterate through [1,2,3,4]. When I use it as Inputs to the For Each frami...
Thank you both very much, I've nailed it . I have accepted Walter_C's answer as solution because Step 2 is what I was missing. Thanks MariuszK as well for your contribution.
Hi!I have a multi-page dashboard based on a handful of tables sharing a common run_id field.I created a dropdown filter widget on the first page to filter datasets based on the table's run_id field.It worked fine for this page, but how to reuse it fo...
Hello @alex456897874, can you try this approach? Create a Global Parameter: Navigate to the Data tab of your dashboard.Define a new parameter based on the run_id field. This parameter will be used globally across all pages. Configure the Filter W...
Do we have same functionality if we use Databricks with Fabric as it provides with Azure?
hi @DEShoaib Are you planing to move Dedicated (T-SQL) pool or Spark code?With Databricks you can replicated all features from Azure Synapse, you have possibility to use PySpark and Databricks SQL. MS Fabric has nice integration with Power BI and eas...
