Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
So I did U2M authentication for account-level operations -databricks auth login --host <account-login-url> --account-id <account-id>then I tried to run code-workspaces = account_client.workspaces.list()workspace_obj=account_client.get_workspace_clien...
I am doing using a service principal so my .databrickscfg has -[databricks-demo]client_id = -----client_secret = ----host = https://accounts.cloud.databricks.com/account_id = ----
Hi Folks, I am looking for any documentation on how Databricks Dashboards handle creating tooltips. Specifically I am interested in dashboards created in the SQL Warehouses (not Workspace Clusters, since using libraries like plotly can have custom to...
Thank you for sharing this. Unfortunately it doesn't really contain much information on how to manage the tooltips that pop up when you hover over a bar or line in a chart/figure. That's what I am looking to customize in (or at least remove from) my ...
I'm trying to use Databricks Apps to host a Streamlit app to serve some interactive applicaton. I face two limitations:1. In environment for App I see it using certain python version, but how to update it to use any other version?It is already set to...
Hi Team good day.I'd like to check on the pattern where dashboards are developed in PowerBI Desktop which are then published to PowerBI Online, using data from Databricks on AWS. In order to refresh the dashboards in PowerBI Online (whether on schedu...
To refresh dashboards in PowerBI Online using data from Databricks on AWS, an on-premises data gateway is not required. This is different from the scenario with Redshift, where an on-premises data gateway is necessary. For Databricks on AWS, you can ...
I am embedding a dashboard in my react application using an iframe.https://www.databricks.com/blog/how-embed-aibi-dashboards-your-websites-and-applicationshttps://docs.databricks.com/en/dashboards/index.html#embed-a-dashboard1. How can i let my users...
Hello @AE10 To allow users to view the dashboard without needing to sign in to Databricks, you can publish the dashboard with embedded credentials. This way, all viewers of the published dashboard can run queries using your credentials for data and ...
Can anyone provide a sample MERGE INTO SQL query for implementing SCD Type 2 in Databricks using Delta Tables?
also, in PySpark, the same example in pyspark:from pyspark.sql.functions import col, concat_ws, current_date, lit, md5source_df = spark.table("source_table")target_df = spark.table("target_table")source_with_hash_df = source_df.withColumn("hash_value...
Hello everyone,I would like to inquire about the possibility of increasing the default limit of concurrent queries on the cluster which is set to 10.While researching this topic, I noticed that there is no official documentation available regarding t...
Did you get an answer how to set spark.databricks.maxConcurrentRuns and spark.databricks.maxConcurrentQueries parameters in sql warehouse serverless cluster?
We have Azure Databricks - SQL Serverless Warehouse with Classic configuration. The config details shared in below snip. As per the Auto Stop config, the Serverless warehouse should be stopped after 10 min of inactivity. However, when we were monitor...
@Digvijay_11 A dbsql warehouse can be triggered not only by a query from a JDBC/ODBC client but also by an idle session, as sessions are managed within the backend clusters. Clusters must be running to maintain these sessions, even before a query is...
Hello,I'm designing a system that combines a lake-house and a real-time path that federates across multiple databases. The requirements led us to adopt Starburst Galaxy (a cloud-hosted distributed query engine) for query federation across the various...
Hello there!So any updates so far? Also interesting in this solution.
I added a filter with the :name_parameter to the dashboard and it works fine.But if I add a date range picker for the date axis it doesn't do any effect (standalone, without the other filter enabled, it is working). It seems to me that the date pick...
Thanks a lot. I guess this was a case of not seeing the forest for the trees. But still, maybe the could add a tooltip or so for this field.
Hi,As a formal requirement in my project I need to keep original, raw (mainly CSVs and XMLs) files on the lake. Later on they are being ingested into Delta format backed medallion stages, bronze, silver, gold etc.From the audit, operations and discov...
Many thanks for raising it @bobmclaren .@Retired_mod Do you have any updates on the above?
We work with a application that connects to our Databricks serverless SQL warehouse via Databricks JDBC driver. It runs a few thousand SQL select statements everyday, and a small percentage of them will fail with the following error details: java.sql...
Hello. Please forgive me if this is not the right place to ask, but I'm having issues with databricks' statement execution api. I'm developing Haskell client for this api.I managed to implement most of it, but I'm running into issues with using named...
Not sure why but I still get the error, even with a simple select when trying to run parametrized sql in a notebook cell:This fails:%pythontable_name = "my_table"%sqlselect org_idfrom identifier(:table_name) While this succeeds:%sqlselect org_idfro...
We are trying to schedule updates to streaming tables and materialized views in Azure Databricks that we have defined in dbt.Two options we are considering are `SCHEDULE CRON` and just scheduling `dbt run` commands via CI/CD. The `SCHEDULE CRON` opti...
Hi, I'd appreciate it if anyone could help!We are using offical ODBC driver (Simba Spark ODBC Driver 64-bit 2.08.02.1013) in our application. C++ APIs.All the following SQL statements are passed through ODBC API to Databricks:successully executing: C...
