Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
I have an on-premises Power BI Report Server that uses the Simba Spark ODBC Driver (2.8) to connect to Databricks. It can connect to a serverless warehouse successfully and run its queries, but it never seems to disconnect the session, and so the war...
Hi @bradleyjamrozik ,The problem is that the current designed behavior of Power BI can lead to the connection not being closed under certain conditions.In short, since Databricks is Async, if an error occurs while fetching data, the connection may no...
I am preparing to migrate from EMR to Databricks and would like to know the best practices for this process. Is there a direct connector, such as a JDBC connector, available to facilitate the migration? Alternatively, would it be more effective to ex...
@Akshay_Petkar if you're planning to migrate from your current technology to Databricks, Travinto Technologies' Code Converter Tool is here to make the process seamless. This powerful tool enables you to migrate data, ETL workflows, and reports acros...
I cancel my subscription plan (account created via accounts.databricks.com), without noticing that by doing this, all of my workspaces will be deleted. Is this possible to recover those workspace, and if possible, what should I do to recover them?
Hello San2. To recover a workspace, could you please raise a case with support? They will be helping you to recover these workspaces, otherwise you can reach your account team, who will help you for further steps, for more details you see disaster ...
Hi all, working on this project, my team plans to migrate some data from some databases to Databricks. We plan to run this migration by submitting queries to a warehouse through python on a local machine.Now I was wondering what would be the best app...
@Ismail1 if you're planning to migrate from your current technology to Databricks, Travinto Technologies' Code Converter Tool is here to make the process seamless. This powerful tool enables you to migrate data, ETL workflows, and reports across pla...
We are using dbt core and running our transformations in a dbt task. Since a few months ago we have started to see a warning message in our runs. This started happening after an update of dbt core and it seems that Databricks needs to update the def...
Both of you missed the sentence "Databricks needs to update the definition of the generated profiles file."The warning is regarding the structure of the profiles.yml which is generated by Databrick. This is not anything I as a user can change accept ...
Hello,I am new to Databricks and was asked to evaluate the new Apps function as a way to publish dashboards. We do not use Databricks today. We have an Azure SQL database we use as a home grown warehouse. The question is can I connect directly to ...
To connect your Azure SQL database to Databricks Apps, you will need to pull the data from the existing database into Databricks. Databricks Apps are designed to leverage the data and features within the Databricks platform, which means that the data...
Hi ,We are looking for a option to copy tables more than 50 TB to be copied from syanpse to databricks on weekly basis , please suggest of there are any feasible ways for samewe are using connector but it is taking too long to copyhttps://learn.micro...
There is no databricks documentation on this as it is only involved for a very tiny bit:"CREATE TABLE catalog.schema.table USING PARQUET LOCATION 'url_to_the_parquet_files'.All the rest is done in Azure Data Factory, or you can even use the built-in ...
So I did U2M authentication for account-level operations -databricks auth login --host <account-login-url> --account-id <account-id>then I tried to run code-workspaces = account_client.workspaces.list()workspace_obj=account_client.get_workspace_clien...
I am doing using a service principal so my .databrickscfg has -[databricks-demo]client_id = -----client_secret = ----host = https://accounts.cloud.databricks.com/account_id = ----
Hi Folks, I am looking for any documentation on how Databricks Dashboards handle creating tooltips. Specifically I am interested in dashboards created in the SQL Warehouses (not Workspace Clusters, since using libraries like plotly can have custom to...
Thank you for sharing this. Unfortunately it doesn't really contain much information on how to manage the tooltips that pop up when you hover over a bar or line in a chart/figure. That's what I am looking to customize in (or at least remove from) my ...
Hi Team good day.I'd like to check on the pattern where dashboards are developed in PowerBI Desktop which are then published to PowerBI Online, using data from Databricks on AWS. In order to refresh the dashboards in PowerBI Online (whether on schedu...
To refresh dashboards in PowerBI Online using data from Databricks on AWS, an on-premises data gateway is not required. This is different from the scenario with Redshift, where an on-premises data gateway is necessary. For Databricks on AWS, you can ...
Hello everyone,I would like to inquire about the possibility of increasing the default limit of concurrent queries on the cluster which is set to 10.While researching this topic, I noticed that there is no official documentation available regarding t...
Did you get an answer how to set spark.databricks.maxConcurrentRuns and spark.databricks.maxConcurrentQueries parameters in sql warehouse serverless cluster?
We have Azure Databricks - SQL Serverless Warehouse with Classic configuration. The config details shared in below snip. As per the Auto Stop config, the Serverless warehouse should be stopped after 10 min of inactivity. However, when we were monitor...
@Digvijay_11 A dbsql warehouse can be triggered not only by a query from a JDBC/ODBC client but also by an idle session, as sessions are managed within the backend clusters. Clusters must be running to maintain these sessions, even before a query is...
Hello,I'm designing a system that combines a lake-house and a real-time path that federates across multiple databases. The requirements led us to adopt Starburst Galaxy (a cloud-hosted distributed query engine) for query federation across the various...
Hello there!So any updates so far? Also interesting in this solution.
I added a filter with the :name_parameter to the dashboard and it works fine.But if I add a date range picker for the date axis it doesn't do any effect (standalone, without the other filter enabled, it is working). It seems to me that the date pick...
Thanks a lot. I guess this was a case of not seeing the forest for the trees. But still, maybe the could add a tooltip or so for this field.
Hi,As a formal requirement in my project I need to keep original, raw (mainly CSVs and XMLs) files on the lake. Later on they are being ingested into Delta format backed medallion stages, bronze, silver, gold etc.From the audit, operations and discov...
Many thanks for raising it @bobmclaren .@Retired_mod Do you have any updates on the above?
