cancel
Showing results for 
Search instead for 
Did you mean: 
Warehousing & Analytics
Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Community. Share insights, tips, and best practices for leveraging data for informed decision-making.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

MadelynM
by Databricks Employee
  • 2989 Views
  • 0 replies
  • 0 kudos

[Recap] Data + AI Summit 2024 - Warehousing & Analytics | Improve performance and increase insights

Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights.  Keynote: Data Warehouse presente...

Screenshot 2024-07-03 at 10.15.26 AM.png
Warehousing & Analytics
AI BI Dashboards
AI BI Genie
Databricks SQL
  • 2989 Views
  • 0 replies
  • 0 kudos
mtreigelman
by New Contributor III
  • 3141 Views
  • 5 replies
  • 5 kudos

Re-Using Datasets inside the Same SQL Dashboard

Hi folks, I am creating a SQL dashboard and want to know if I can re-use datasets within the same dashboard. The screenshot below captures what I would like to do pretty well, but to summarize... I need to run an computationally expensive query and w...

databricks_dash_dataset_recycle_question.png
  • 3141 Views
  • 5 replies
  • 5 kudos
Latest Reply
mtreigelman
New Contributor III
  • 5 kudos

@rskuntoji24 My current workaround has been to create the first query as a materialized view in my catalog, and then run a workflow daily to update the view. The downsides to this strategy is that it might not be practical for streaming data (not an ...

  • 5 kudos
4 More Replies
QueryingQuagga
by New Contributor III
  • 3541 Views
  • 1 replies
  • 0 kudos

Databricks SQL editor - we can get a long way with a few Quality of Life improvements

Hello all (and, if you are watching these posts, Databricks employees/representatives ),I hope you take this idea into account - I haven't been able to find a place where one can vote on features, but I believe that this idea would make very real im...

  • 3541 Views
  • 1 replies
  • 0 kudos
Latest Reply
QueryingQuagga
New Contributor III
  • 0 kudos

Although this is not getting any visibility, I've got another feature idea to add:* Colour coding or otherwise marking columns to reflect table relationIn a simple join between e.g. table "cart" and table "cartItems", columns from "cart" would be mar...

  • 0 kudos
NikkG
by New Contributor
  • 1382 Views
  • 3 replies
  • 0 kudos

Worker/Drive Node Config for Free Tier EastUS region

Hello,Right now I have chosen the free tier mode and region is east US. But I am unable to create a all purpose compute cluster. Any config I chose it says not supported. Any help is appreciated for pointing which config for worker and driver I shoul...

  • 1382 Views
  • 3 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

It seems to be an issue with instance availability on the zone, it could be that at the given time there were no infrastructure available or a limitation with the zone itself. Have you checked the link available in the error message?  

  • 0 kudos
2 More Replies
HeathDG1
by New Contributor II
  • 960 Views
  • 1 replies
  • 0 kudos

Display Pyspark Widget in UI Dashboard

Hi-I have a spark DF that I create a visual from, I added a dbutils widget to filter the visual. I want to display the visual in our current IU dashboards (that were created with SQL datasets and calculated measures) but when I click on the drop down...

  • 960 Views
  • 1 replies
  • 0 kudos
Latest Reply
Advika_
Databricks Employee
  • 0 kudos

Hello @HeathDG1! It looks like your visualization was created in a Databricks Notebook using the dbutils widget for filtering. However, Databricks only allows SQL-based visualizations to be added to AI/BI dashboards, while notebook visuals can only b...

  • 0 kudos
VCA50380
by Contributor II
  • 2026 Views
  • 2 replies
  • 0 kudos

Resolved! Error "[INSUFFICIENT_PERMISSIONS] Insufficient privileges SQLSTATE: 42501"

Hello,When I do some DML statements using a view "the_source_view"  direcly in SQL as a basis, all works fine. I can do inserts in another Delta table based on the data coming out of this view.It works too from a Job: WorkFlow -> Task -> Notebook con...

  • 2026 Views
  • 2 replies
  • 0 kudos
Latest Reply
VCA50380
Contributor II
  • 0 kudos

Hello,Things are working as soon as I use a compute with Access Mode = Standard.Vinc

  • 0 kudos
1 More Replies
HeathDG1
by New Contributor II
  • 1081 Views
  • 2 replies
  • 0 kudos

Resolved! Dashboard Widget Alignment

Hi, Is there a way to aline the databricks widget filters. Please see screenshot below. 

  • 1081 Views
  • 2 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hello @HeathDG1, Looks like screenshot is cut off, could you please elaborate more on what do you want to achieve?

  • 0 kudos
1 More Replies
Torch3333
by New Contributor II
  • 1284 Views
  • 1 replies
  • 1 kudos

Resolved! Linearizability on Delta Lake table

Hi all, Does Delta Lake table guarantee linearizability for the following operations on a single record:- SELECT- UPDATE and DELETE with condition (WHERE clauses) If linearizability is not guaranteed, what consistency model does Delta Lake provide fo...

  • 1284 Views
  • 1 replies
  • 1 kudos
Latest Reply
Advika
Databricks Employee
  • 1 kudos

Hello @Torch3333! Delta Lake does not guarantee linearizability for single record operations. In Delta Lake, isolation levels ensure consistency guarantees in transactions. By default, SELECT operations follow snapshot isolation, ensuring that reads ...

  • 1 kudos
mausch
by New Contributor
  • 2970 Views
  • 0 replies
  • 0 kudos

CalledProcessError when running dbt

I've been trying to run a dbt project (sourced in Azure DevOps) in Databricks Workflows, but I get this error message:  CalledProcessError: Command 'b'\nmkdir -p "/tmp/tmp-dbt-run-1124228490001263"\nunexpected_errors="$(cp -a -u "/Workspace/Repos/.in...

  • 2970 Views
  • 0 replies
  • 0 kudos
edejong1980
by New Contributor II
  • 3206 Views
  • 1 replies
  • 1 kudos

Databricks SQL Warehouse does not scale down to 0

Hey Support Team,We are experiencing an issue with our Databricks warehouse where the auto-stop feature does not seem to be working as expected. Despite setting an idle timeout, the warehouse continues running after the configured auto-stop time has ...

  • 3206 Views
  • 1 replies
  • 1 kudos
Latest Reply
edejong1980
New Contributor II
  • 1 kudos

We were able to diagnose and resolve the problem. The problem was caused due to a cube.js JDBC connection repeatedly connecting to our Databricks SQL Warehouse. Databricks SQL Warehouse does not scale down when there are repeated new connections made...

  • 1 kudos
sunnypol
by New Contributor
  • 3288 Views
  • 1 replies
  • 0 kudos

SQL Warehouse: INVALID_PARAMETER_VALUE when starting

Hi Team,Suddenly our SQL warehouse stopped running, can you please help us how can we check why it stopped.  

  • 3288 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ismael-K
Databricks Employee
  • 0 kudos

A similar error was reported, please check this thread to see if it helps resolve the issue.

  • 0 kudos
Akshay_Petkar
by Valued Contributor
  • 1328 Views
  • 2 replies
  • 2 kudos

Resolved! Row-Level Security Not Working in Published Databricks AI/BI Dashboard

I have applied row-level security (RLS) on the department column in the table so that users can only see data related to their own department. The security policy works perfectly when I query the table in Databricks SQL.Now, I have built a Databricks...

  • 1328 Views
  • 2 replies
  • 2 kudos
Latest Reply
koji_kawamura
Databricks Employee
  • 2 kudos

Hi @Akshay_Petkar , which credential mode did you use to publish the dashboard? In order to make access control work based on the viewer's credential, "Don't embed credentials" mode should be used. If you published the dashboard with "Embed credentia...

  • 2 kudos
1 More Replies
VCA50380
by Contributor II
  • 1536 Views
  • 3 replies
  • 0 kudos

Resolved! Equivalent of Oracle's CLOB in Databricks

Dear all,(migrating for an on-premise Oracle ...)The question is in the subject: "What is the equivalent of Oracle's CLOB in Databricks" ?I saw that the "string" type can go up to 50 thousands characters, which is quite good in most of our cases, but...

  • 1536 Views
  • 3 replies
  • 0 kudos
Latest Reply
VCA50380
Contributor II
  • 0 kudos

Hello;Thanks for the answer.For the concatenation itself, it is not an issue.My question is "is Databricks supporting something bigger than the 'string' data-type" ? Thanks

  • 0 kudos
2 More Replies
VCA50380
by Contributor II
  • 3468 Views
  • 10 replies
  • 3 kudos

Resolved! Efficacy of PySpark in Databricks

Hi all,- migrating from an on-premise Oracle -Currently on Oracle, I have a "library" of let's say 300 tables to load, sequentially, based on views (some tables being fed potentially by several views, therefore the number of underlying views is highe...

  • 3468 Views
  • 10 replies
  • 3 kudos
Latest Reply
MariuszK
Valued Contributor III
  • 3 kudos

The common scenario for data processing in Oracle is based on PL/SQL and cursors. In the case of PySpark we don't have such concept as cursors and iteration on data frames can lead to poor performance.I migrated Oracle to Databricks and I learned tha...

  • 3 kudos
9 More Replies
bcb44
by New Contributor
  • 433 Views
  • 1 replies
  • 0 kudos

How to write a custom datasource for shared (standard access mode)

Hello,I maintain a spark plugin with users who are moving from spark to databricks and want to use shared access mode. My library has a bunch of custom datasources and users are seeing errors when using them. Is there a way to write a custom datasour...

  • 433 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hello @bcb44, Thanks for your question.  Data Source V2 Relations: These are not supported in clusters that are configured with Table ACL or Credential Passthrough. Can you validate if ACL or Passthrough are enabled on the cluster? Also this should w...

  • 0 kudos