cancel
Showing results for 
Search instead for 
Did you mean: 
Warehousing & Analytics
Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Community. Share insights, tips, and best practices for leveraging data for informed decision-making.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

MadelynM
by Databricks Employee
  • 1966 Views
  • 0 replies
  • 0 kudos

[Recap] Data + AI Summit 2024 - Warehousing & Analytics | Improve performance and increase insights

Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights.  Keynote: Data Warehouse presente...

Screenshot 2024-07-03 at 10.15.26 AM.png
Warehousing & Analytics
AI BI Dashboards
AI BI Genie
Databricks SQL
  • 1966 Views
  • 0 replies
  • 0 kudos
dougtrajano
by New Contributor
  • 81 Views
  • 2 replies
  • 0 kudos

How does the AI/BI Dashboard Custom Calculations feature actually work?

I am trying to compute the market_share measure using the Custom Calculations functionality in the AI/BI Dashboard.My dataset looks like this:empresa (company): The name of the company.acessos (accesses): Count of access.My Custom Calculation express...

Screenshot 2025-05-18 at 12.00.04 PM.png Screenshot 2025-05-18 at 12.00.37 PM.png
  • 81 Views
  • 2 replies
  • 0 kudos
Latest Reply
Shua42
Databricks Employee
  • 0 kudos

Hi @dougtrajano , The idea of the custom calculations here are that they evaluate to a single value for the entire table. In the expression here, you are trying to divide a column (acessos) by a single value (sum of acessos). The error is happening b...

  • 0 kudos
1 More Replies
DEShoaib
by New Contributor II
  • 1675 Views
  • 7 replies
  • 1 kudos

Fabric with Databricks

Do we have same functionality if we use Databricks with Fabric as it provides with Azure?

  • 1675 Views
  • 7 replies
  • 1 kudos
Latest Reply
dks
New Contributor
  • 1 kudos

We aim to implement Databricks Mirroring through the Fabric APIs for automation. However, the Mirroring API specifically states that it is not compatible with Databricks. Are there alternative APIs that could be used to achieve this functionality?

  • 1 kudos
6 More Replies
Dave_Nithio
by Contributor II
  • 1109 Views
  • 1 replies
  • 0 kudos

Monitoring Databricks SQL Warehouse without Unity Catalog

I am looking to monitor my SQL Warehouse, especially the 'Running Clusters' metric that is available in the monitoring tab of the warehouse. This shows the start and shut down time as well as the number of running clusters:The issue I have run into i...

Dave_Nithio_0-1741034562647.png
  • 1109 Views
  • 1 replies
  • 0 kudos
Latest Reply
Isi
Contributor III
  • 0 kudos

Hey @Dave_Nithio To monitor the “Running Clusters” metric for your SQL Warehouse, you can use the Databricks Cluster Events API. This API retrieves a list of events related to cluster activity, such as start and shutdown times, and provides paginated...

  • 0 kudos
barchiel33
by New Contributor II
  • 620 Views
  • 3 replies
  • 1 kudos

Databricks SQL Python - Result fetching takes extremely long time

Hello All!I have a python script which utilizes the databricks SQL for python package in order to pull a databricks table into a pandas dataframe which is used to create a table in a Spotfire report. The table contains ~1.28 million rows, with 155 co...

  • 620 Views
  • 3 replies
  • 1 kudos
Latest Reply
Isi
Contributor III
  • 1 kudos

Hey @barchiel33 ,After reviewing your context further, I believe the most effective approach would be to set up an automated pipeline within Databricks that periodically extracts data based on the frequency you need (daily, weekly, hourly, etc.), cre...

  • 1 kudos
2 More Replies
ossinova
by Contributor II
  • 1030 Views
  • 2 replies
  • 0 kudos

Coss page filters - AI/BI Dasboards

When creating a dashboard with multiple pages connected to one dataset. It seems that only visual elements on the same page that the filter is on takes effect. Is there a way to filter all visual elements regardless of which page the filter is on?I h...

  • 1030 Views
  • 2 replies
  • 0 kudos
Latest Reply
SP_6721
New Contributor III
  • 0 kudos

It seems that, filter can be applied to only to visualizations on the same page that share the same dataset. Currently, there are no a built in features to create a global filter that works on multiple pages within a dashboard.To achieve filtering ac...

  • 0 kudos
1 More Replies
alex456897874
by New Contributor III
  • 1295 Views
  • 7 replies
  • 1 kudos

how to reuse filter between pages in dashboards?

Hi!I have a multi-page dashboard based on a handful of tables sharing a common run_id field.I created a dropdown filter widget on the first page to filter datasets based on the table's run_id field.It worked fine for this page, but how to reuse it fo...

  • 1295 Views
  • 7 replies
  • 1 kudos
Latest Reply
Benjamin_DavidS
New Contributor II
  • 1 kudos

I have the same problem here. filters are not applied while switching pages

  • 1 kudos
6 More Replies
Ajbi
by New Contributor II
  • 104 Views
  • 0 replies
  • 0 kudos

Power BI refresh from Databricks workflow

We recently enabled the preview feature to refresh power bi datasets from data bricks workflow, but when we run the job data changes aren't being applied despite a completed status in refresh history. Could anyone clarify whether this is intended pre...

Warehousing & Analytics
Databricks Power BI Task
power bi refresh
  • 104 Views
  • 0 replies
  • 0 kudos
tbmadhav
by New Contributor
  • 144 Views
  • 1 replies
  • 0 kudos

Convert a delta table (flattened json format) to a nested java object

I have a delta table with columns like below format. I am querying Databricks using databricks-jdbc 2.7.1 driver. I want to convert the result set to a nested java object.Order_actualPickupAddress_residential -> stringOrder_pickupRequestDetails -> ar...

  • 144 Views
  • 1 replies
  • 0 kudos
Latest Reply
BigRoux
Databricks Employee
  • 0 kudos

Here are some suggestions/ideas to consider:   To map hundreds of nested fields from a Databricks Delta table to complex Java objects like the Order object described, consider the following approaches:   1. Using Libraries for Object Mapping Utilize ...

  • 0 kudos
pavel_cerny
by New Contributor II
  • 2363 Views
  • 2 replies
  • 1 kudos

Databricks JDBC driver fails with socket read timeout

Application connects to Databricks serverless SQL warehouse via Databricks JDBC driver. It executes SQL select statements only. We see small number of statements failed each day with the following error detail: java.sql.SQLException: [Databricks][JDB...

  • 2363 Views
  • 2 replies
  • 1 kudos
Latest Reply
kamal_ch
Databricks Employee
  • 1 kudos

Interesting issue, looks like a Cluster Load issue. Recommend filing a support ticket with Databticks to investigate, in case if you haven't already

  • 1 kudos
1 More Replies
jakobhaggstrom
by New Contributor
  • 821 Views
  • 2 replies
  • 0 kudos

Issue when using M2M authentication with azure databricks jdbc driver 2.7.1

Hi!I try to connect to an azure databricks sql warehouse in dbeaver, which uses the azure databricks jdbc driver version 2.7.1 ## and I cannot get M2M authentication to work. I get a 'Not Authorized' (401) response when I try to connect, and it seems...

  • 821 Views
  • 2 replies
  • 0 kudos
Latest Reply
Renu_
Contributor
  • 0 kudos

Hi @jakobhaggstrom, this error likely occurs due to the type of secret you're using. For M2M authentication, the Databricks JDBC driver requires a Databricks generated OAuth secret, not a Microsoft Entra ID client secret. While your service principal...

  • 0 kudos
1 More Replies
Akshay_Petkar
by Contributor III
  • 1005 Views
  • 1 replies
  • 0 kudos

Best Approach to Connect Quick Book with Databricks

What’s the best and seamless way to connect Quick Books data to Databricks? Is there any recommended third-party tool or direct connection using JDBC/ODBC?

  • 1005 Views
  • 1 replies
  • 0 kudos
Latest Reply
kamal_ch
Databricks Employee
  • 0 kudos

Hi Akshay,   You can use third-party tools that specialize in integrating with external platforms like Databricks. Tools such as CData, QODBC, or similar database connectors provide JDBC and ODBC drivers for QB. These drivers allow you to query Quick...

  • 0 kudos
lucy-ji
by New Contributor II
  • 489 Views
  • 5 replies
  • 1 kudos

Resolved! How to download widget result into CSV

In the databricks Tutorial (https://docs.databricks.com/gcp/en/sql/get-started/sample-dashboards) , it says we can right click the widget and choose "Download as CSV".  However, in my databricks dashboard, when I right click the widgets, there is no ...

  • 489 Views
  • 5 replies
  • 1 kudos
Latest Reply
lucy-ji
New Contributor II
  • 1 kudos

I find the reason is that my Admin didn't enable "SQL result download" option. Now it is solved. Thanks ALL.

  • 1 kudos
4 More Replies
Kaz
by New Contributor II
  • 7779 Views
  • 3 replies
  • 1 kudos

Show full logs on job log

Is it possible to show the full logs of a databricks job? Currently, the logs are skipped with:*** WARNING: max output size exceeded, skipping output. ***However, I don't believe our log files are more than 20 MB. I know you can press the logs button...

  • 7779 Views
  • 3 replies
  • 1 kudos
Latest Reply
Isi
Contributor III
  • 1 kudos

Hey @Kaz ,Unfortunately, the output truncation limit in the Databricks job UI cannot be changed. Once that limit is exceeded, the rest of the logs are skipped, and the full logs become accessible only through the “Logs” button, which, as you mentione...

  • 1 kudos
2 More Replies
teixeire
by New Contributor II
  • 421 Views
  • 3 replies
  • 0 kudos

How to restrict external access to SQL Warehouse but allow workspace queries?

Hi everyone,I'm currently setting up access controls in our Databricks development workspace. The goal is to enable business users to explore data and build their SQL skills within the workspace itself (e.g., via SQL editor or notebooks), but prevent...

Warehousing & Analytics
Access Rights
Endpoint
sql
  • 421 Views
  • 3 replies
  • 0 kudos
Latest Reply
Isi
Contributor III
  • 0 kudos

Hi @teixeire ,To prevent external tools like Power BI or DBeaver from connecting to your SQL Warehouse, one effective approach is to restrict personal access token (PAT) creation for users who should only query data inside the Databricks workspace.Th...

  • 0 kudos
2 More Replies
meghana_tulla
by New Contributor III
  • 543 Views
  • 5 replies
  • 0 kudos

UCX V0.58.0 Dashboard in Databricks Showing 0 Tables Despite Existing Tables.

Hi,I am currently installing UCX version 0.58.0 in Non-Unity enabled Databricks Workspace. It installed successfully and the job completes without errors. However, The Dashboard that goanna generate showing "0 Total Tables" and other metrics as 0.Has...

  • 543 Views
  • 5 replies
  • 0 kudos
Latest Reply
meghana_tulla
New Contributor III
  • 0 kudos

Hi,Still facing the same issue actually, I am trying to automate the UCX installation using bashscript. while running the bashscript the job starts and run successfully with all run jobs. Even all jobs Ran successfully, I am getting 0 tables and 0 va...

  • 0 kudos
4 More Replies