cancel
Showing results for 
Search instead for 
Did you mean: 
Warehousing & Analytics
Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Community. Share insights, tips, and best practices for leveraging data for informed decision-making.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

MadelynM
by Databricks Employee
  • 1238 Views
  • 0 replies
  • 0 kudos

[Recap] Data + AI Summit 2024 - Warehousing & Analytics | Improve performance and increase insights

Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights.  Keynote: Data Warehouse presente...

Screenshot 2024-07-03 at 10.15.26 AM.png
Warehousing & Analytics
AI BI Dashboards
AI BI Genie
Databricks SQL
  • 1238 Views
  • 0 replies
  • 0 kudos
HeathDG1
by New Contributor II
  • 271 Views
  • 1 replies
  • 0 kudos

Row filtering based on condition not working

Hi-We have a delta table in our unity catalog called dream_team.stern_portfolio.location_info.We are trying to use row level security to filter our data based on a users group membership. This way when users look at out dashboard they can only see th...

  • 271 Views
  • 1 replies
  • 0 kudos
Latest Reply
Isi
Contributor
  • 0 kudos

Hey @HeathDG1 I think your function isn’t behaving the way you expect because of how the logic is set up:•If the user is in Stern MA, they only see rows where state = 'MA'  (which is good).•BUT for everyone else, the function returns true, meaning th...

  • 0 kudos
fishingrod
by New Contributor II
  • 319 Views
  • 3 replies
  • 0 kudos

How to implement automatic scaling of cluster size in Serverless Warehouse

I would like to know if the cluster size of a Serverless Warehouse can automatically scale up and down, and what determines the number of workers used when executing queries. Does it use all workers within the cluster size fixedly, or does it use par...

  • 319 Views
  • 3 replies
  • 0 kudos
Latest Reply
Takuya-Omi
Valued Contributor II
  • 0 kudos

@fishingrod My understanding is that Intelligent Workload Management (IWM) in Serverless SQL Warehouses adjusts the number of clusters, but it does not automatically scale the cluster size.This means that if you need to improve the execution performa...

  • 0 kudos
2 More Replies
THIAM_HUATTAN
by Valued Contributor
  • 2312 Views
  • 3 replies
  • 0 kudos

Visualization from Python dataframe?

I notice it is very easily to get visualization from sql language inside Databricks. Say you run a SQL query which gives you a table, and you can easily use that table to do its visualization in terms of plots.How about in Python language when we hav...

  • 2312 Views
  • 3 replies
  • 0 kudos
Latest Reply
KenChase99
New Contributor II
  • 0 kudos

 Yes! You can visualize a Python DataFrame in Databricks easily using: display(df)This works like SQL visualizations, offering built-in charts. For more customization, Matplotlib, Seaborn, or Plotly can be used. Would love to see even more native sup...

  • 0 kudos
2 More Replies
nirquant
by New Contributor
  • 462 Views
  • 0 replies
  • 0 kudos

VS Code Extension Sync Destination Changed Unexpectedly

I recently updated my Databricks VS Code extension from version 1.4.0 to 2.4.0. After the update, I noticed that:The "Configure Sync Destination" button is missing.The sync destination format has changed to a remote folder instead of the previous set...

  • 462 Views
  • 0 replies
  • 0 kudos
DataFarmer
by New Contributor II
  • 5713 Views
  • 5 replies
  • 1 kudos

Resolved! How to let Business Users edit tables in Databricks

Hi Community!I have the requirement that business users shall be able to edit/update tables in Unity Catalog, e.g. master data records, mapping tables. I also want thes actions to be logged for auditing/troubleshooting.Is there any simple solution to...

  • 5713 Views
  • 5 replies
  • 1 kudos
Latest Reply
kenwong
Databricks Employee
  • 1 kudos

We do have a few partners that offer solutions in this space (e.g. Retool).  Recently, Sigma added support for their InputTable feature which was designed for this use case: https://www.sigmacomputing.com/blog/bring-your-own-data-to-databricks-with-s...

  • 1 kudos
4 More Replies
Akshay_Petkar
by Contributor III
  • 1218 Views
  • 5 replies
  • 2 kudos

How to Display Top Categories in Databricks AI/BI Dashboard?

In a Databricks AI/BI dashboard, I have a field with multiple categories (e.g., district-wise sales with 50 districts). How can I display only the top few categories (like the top 10) based on a specific metric such as sales?

  • 1218 Views
  • 5 replies
  • 2 kudos
Latest Reply
Mo
Databricks Employee
  • 2 kudos

hey @migq2 , @maks  in the AI/BI dashboards in your data, add a limit parameter like:select all from my_table limit :limit_number to all your tables. when you're on canvas and adding visualizations, add a filter and create a parameter with single val...

  • 2 kudos
4 More Replies
Akshay_Petkar
by Contributor III
  • 209 Views
  • 1 replies
  • 0 kudos

How to get the size of selected rows in bytes using a single SQL query?

Hi all,I have a table named employee in Databricks. I ran the following query to filter out rows where the salary is greater than 25000.This query returns 10 rows. I want to find the size of these 10 rows in bytes, and I would like to calculate or re...

  • 209 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @Akshay_Petkar, You can try with this query: SELECT SUM(LENGTH(CAST(employee.* AS STRING))) AS total_size_in_bytesFROM employeeWHERE salary > 25000;

  • 0 kudos
vidya_kothavale
by New Contributor III
  • 396 Views
  • 1 replies
  • 1 kudos

Resolved! Insufficient Permissions Error When Reading Data from S3 in Shared Databricks Compute

I am using a Shared Databricks Compute and trying to read data from an S3 bucket via an Instance Profile. However, I am encountering the following error: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission SELECT on any ...

  • 396 Views
  • 1 replies
  • 1 kudos
Latest Reply
Ayushi_Suthar
Databricks Employee
  • 1 kudos

Hi @vidya_kothavale , Greetings! Can you please refer to this article and check if it helps you to resolve your issue : https://kb.databricks.com/en_US/data/user-does-not-have-permission-select-on-any-file Please note that these permissions are only ...

  • 1 kudos
Akshay_Petkar
by Contributor III
  • 299 Views
  • 2 replies
  • 1 kudos

Databricks Dashboard Not Refreshing as per Schedule (Per Minute)

I have created a Ai/Bi dashboard on Databricks and set it to refresh every 1 minute. However, the dashboard is not refreshing as expected. Interestingly, when I terminate the warehouse, it triggers the warehouse but still does not refresh per minute ...

  • 299 Views
  • 2 replies
  • 1 kudos
Latest Reply
Akshay_Petkar
Contributor III
  • 1 kudos

Hi @Alberto_Umana ,I have followed the correct approach to create and schedule the dashboard for a 1-minute refresh, but it's not updating every minute as expected.I have attached images for your reference. Please take a look.

  • 1 kudos
1 More Replies
onlyme
by New Contributor II
  • 345 Views
  • 2 replies
  • 1 kudos

Resolved! Actions for warehouse channel update

Hello Let's say I create a SQL Warehouse on Current Channel (2024.40) and there is a new release (2024.50).Would I need to take some actions(reboot for example) so that my warehouse uses the 2024.50 version or it should run on the 2024.50 whenever th...

  • 345 Views
  • 2 replies
  • 1 kudos
Latest Reply
Isi
Contributor
  • 1 kudos

Hey @onlyme ,The Channel in Databricks SQL Warehouse has two options:1.Current: This corresponds to the latest stable version released by Databricks and updates automatically.2.Preview: Similar to a beta version, it includes improvements and new feat...

  • 1 kudos
1 More Replies
GarCab
by New Contributor
  • 361 Views
  • 1 replies
  • 0 kudos

Databricks connectivity issue with PBI service

Hello everyone,I created a report using PowerBI Desktop that I successfully connected to Databricks. However, in PBI Service, the visuals are not displayed and I'm asked to edit credentials of the semantic model. When doing so, I get the following er...

  • 361 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @GarCab, Can you try with a PAT token, just to confirm it works? https://learn.microsoft.com/en-us/azure/databricks/partners/bi/power-bi#connect-power-bi-desktop-to-azure-databricks

  • 0 kudos
Hania__b
by New Contributor II
  • 414 Views
  • 3 replies
  • 2 kudos

Resolved! Access specific input item of For Each Tasks

Hi,I think I have a similar issue to the one in this post, but the answer isn't detailed enough for me.I have a list defined in my first task, which contains the items I want to iterate through [1,2,3,4]. When I use it as Inputs to the For Each frami...

  • 414 Views
  • 3 replies
  • 2 kudos
Latest Reply
Hania__b
New Contributor II
  • 2 kudos

Thank you both very much, I've nailed it . I have accepted Walter_C's answer as solution because Step 2 is what I was missing. Thanks MariuszK as well for your contribution.

  • 2 kudos
2 More Replies
alex456897874
by New Contributor II
  • 185 Views
  • 1 replies
  • 0 kudos

how to reuse filter between pages in dashboards?

Hi!I have a multi-page dashboard based on a handful of tables sharing a common run_id field.I created a dropdown filter widget on the first page to filter datasets based on the table's run_id field.It worked fine for this page, but how to reuse it fo...

  • 185 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hello @alex456897874, can you try this approach?   Create a Global Parameter: Navigate to the Data tab of your dashboard.Define a new parameter based on the run_id field. This parameter will be used globally across all pages. Configure the Filter W...

  • 0 kudos
DEShoaib
by New Contributor II
  • 786 Views
  • 6 replies
  • 1 kudos

Fabric with Databricks

Do we have same functionality if we use Databricks with Fabric as it provides with Azure?

  • 786 Views
  • 6 replies
  • 1 kudos
Latest Reply
MariuszK
Contributor III
  • 1 kudos

hi @DEShoaib Are you planing to move Dedicated (T-SQL) pool or Spark code?With Databricks you can replicated all features from Azure Synapse, you have possibility to use PySpark and Databricks SQL. MS Fabric has nice integration with Power BI and eas...

  • 1 kudos
5 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels
Top Kudoed Authors