Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
Hello,Right now I have chosen the free tier mode and region is east US. But I am unable to create a all purpose compute cluster. Any config I chose it says not supported. Any help is appreciated for pointing which config for worker and driver I shoul...
It seems to be an issue with instance availability on the zone, it could be that at the given time there were no infrastructure available or a limitation with the zone itself. Have you checked the link available in the error message?
Hi-I have a spark DF that I create a visual from, I added a dbutils widget to filter the visual. I want to display the visual in our current IU dashboards (that were created with SQL datasets and calculated measures) but when I click on the drop down...
Hello @HeathDG1! It looks like your visualization was created in a Databricks Notebook using the dbutils widget for filtering. However, Databricks only allows SQL-based visualizations to be added to AI/BI dashboards, while notebook visuals can only b...
Is it possible to show the full logs of a databricks job? Currently, the logs are skipped with:*** WARNING: max output size exceeded, skipping output. ***However, I don't believe our log files are more than 20 MB. I know you can press the logs button...
Hey KazWhere did you get that ?
Hello,When I do some DML statements using a view "the_source_view" direcly in SQL as a basis, all works fine. I can do inserts in another Delta table based on the data coming out of this view.It works too from a Job: WorkFlow -> Task -> Notebook con...
Hello,Things are working as soon as I use a compute with Access Mode = Standard.Vinc
When creating a dashboard with multiple pages connected to one dataset. It seems that only visual elements on the same page that the filter is on takes effect. Is there a way to filter all visual elements regardless of which page the filter is on?I h...
Hi, Is there a way to aline the databricks widget filters. Please see screenshot below.
Hello @HeathDG1, Looks like screenshot is cut off, could you please elaborate more on what do you want to achieve?
Hi all, Does Delta Lake table guarantee linearizability for the following operations on a single record:- SELECT- UPDATE and DELETE with condition (WHERE clauses) If linearizability is not guaranteed, what consistency model does Delta Lake provide fo...
Hello @Torch3333! Delta Lake does not guarantee linearizability for single record operations. In Delta Lake, isolation levels ensure consistency guarantees in transactions. By default, SELECT operations follow snapshot isolation, ensuring that reads ...
I've been trying to run a dbt project (sourced in Azure DevOps) in Databricks Workflows, but I get this error message: CalledProcessError: Command 'b'\nmkdir -p "/tmp/tmp-dbt-run-1124228490001263"\nunexpected_errors="$(cp -a -u "/Workspace/Repos/.in...
I am going through the labs for the CDW Migration Best Practices training on the Databricks Learning Portal and scored 75%. However, it still shows as "In Progress."Can anyone explain the criteria for marking the labs as complete? Do I need a higher ...
Hello,I'm trying to finish Delivery Specialization: CDW Migration Best Practices course but I could able to achieve only 50%.could you please share those answer notebooks to this email "betterahyosi2k20@gmail.com" ???
Hey Support Team,We are experiencing an issue with our Databricks warehouse where the auto-stop feature does not seem to be working as expected. Despite setting an idle timeout, the warehouse continues running after the configured auto-stop time has ...
We were able to diagnose and resolve the problem. The problem was caused due to a cube.js JDBC connection repeatedly connecting to our Databricks SQL Warehouse. Databricks SQL Warehouse does not scale down when there are repeated new connections made...
Hi Team,Suddenly our SQL warehouse stopped running, can you please help us how can we check why it stopped.
A similar error was reported, please check this thread to see if it helps resolve the issue.
I have applied row-level security (RLS) on the department column in the table so that users can only see data related to their own department. The security policy works perfectly when I query the table in Databricks SQL.Now, I have built a Databricks...
Hi @Akshay_Petkar , which credential mode did you use to publish the dashboard? In order to make access control work based on the viewer's credential, "Don't embed credentials" mode should be used. If you published the dashboard with "Embed credentia...
Dear all,(migrating for an on-premise Oracle ...)The question is in the subject: "What is the equivalent of Oracle's CLOB in Databricks" ?I saw that the "string" type can go up to 50 thousands characters, which is quite good in most of our cases, but...
Hello;Thanks for the answer.For the concatenation itself, it is not an issue.My question is "is Databricks supporting something bigger than the 'string' data-type" ? Thanks
Hi all,- migrating from an on-premise Oracle -Currently on Oracle, I have a "library" of let's say 300 tables to load, sequentially, based on views (some tables being fed potentially by several views, therefore the number of underlying views is highe...
The common scenario for data processing in Oracle is based on PL/SQL and cursors. In the case of PySpark we don't have such concept as cursors and iteration on data frames can lead to poor performance.I migrated Oracle to Databricks and I learned tha...
I am looking to monitor my SQL Warehouse, especially the 'Running Clusters' metric that is available in the monitoring tab of the warehouse. This shows the start and shut down time as well as the number of running clusters:The issue I have run into i...
