Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
Hi ,We are looking for a option to copy tables more than 50 TB to be copied from syanpse to databricks on weekly basis , please suggest of there are any feasible ways for samewe are using connector but it is taking too long to copyhttps://learn.micro...
There is no databricks documentation on this as it is only involved for a very tiny bit:"CREATE TABLE catalog.schema.table USING PARQUET LOCATION 'url_to_the_parquet_files'.All the rest is done in Azure Data Factory, or you can even use the built-in ...
So I did U2M authentication for account-level operations -databricks auth login --host <account-login-url> --account-id <account-id>then I tried to run code-workspaces = account_client.workspaces.list()workspace_obj=account_client.get_workspace_clien...
I am doing using a service principal so my .databrickscfg has -[databricks-demo]client_id = -----client_secret = ----host = https://accounts.cloud.databricks.com/account_id = ----
Hi Folks, I am looking for any documentation on how Databricks Dashboards handle creating tooltips. Specifically I am interested in dashboards created in the SQL Warehouses (not Workspace Clusters, since using libraries like plotly can have custom to...
Thank you for sharing this. Unfortunately it doesn't really contain much information on how to manage the tooltips that pop up when you hover over a bar or line in a chart/figure. That's what I am looking to customize in (or at least remove from) my ...
Hi Team good day.I'd like to check on the pattern where dashboards are developed in PowerBI Desktop which are then published to PowerBI Online, using data from Databricks on AWS. In order to refresh the dashboards in PowerBI Online (whether on schedu...
To refresh dashboards in PowerBI Online using data from Databricks on AWS, an on-premises data gateway is not required. This is different from the scenario with Redshift, where an on-premises data gateway is necessary. For Databricks on AWS, you can ...
Hello everyone,I would like to inquire about the possibility of increasing the default limit of concurrent queries on the cluster which is set to 10.While researching this topic, I noticed that there is no official documentation available regarding t...
Did you get an answer how to set spark.databricks.maxConcurrentRuns and spark.databricks.maxConcurrentQueries parameters in sql warehouse serverless cluster?
We have Azure Databricks - SQL Serverless Warehouse with Classic configuration. The config details shared in below snip. As per the Auto Stop config, the Serverless warehouse should be stopped after 10 min of inactivity. However, when we were monitor...
@Digvijay_11 A dbsql warehouse can be triggered not only by a query from a JDBC/ODBC client but also by an idle session, as sessions are managed within the backend clusters. Clusters must be running to maintain these sessions, even before a query is...
Hello,I'm designing a system that combines a lake-house and a real-time path that federates across multiple databases. The requirements led us to adopt Starburst Galaxy (a cloud-hosted distributed query engine) for query federation across the various...
Hello there!So any updates so far? Also interesting in this solution.
I added a filter with the :name_parameter to the dashboard and it works fine.But if I add a date range picker for the date axis it doesn't do any effect (standalone, without the other filter enabled, it is working). It seems to me that the date pick...
Thanks a lot. I guess this was a case of not seeing the forest for the trees. But still, maybe the could add a tooltip or so for this field.
Hi,As a formal requirement in my project I need to keep original, raw (mainly CSVs and XMLs) files on the lake. Later on they are being ingested into Delta format backed medallion stages, bronze, silver, gold etc.From the audit, operations and discov...
Many thanks for raising it @bobmclaren .@Retired_mod Do you have any updates on the above?
We are trying to schedule updates to streaming tables and materialized views in Azure Databricks that we have defined in dbt.Two options we are considering are `SCHEDULE CRON` and just scheduling `dbt run` commands via CI/CD. The `SCHEDULE CRON` opti...
Hi all, I am currently stuck because runing DESCRIBE DETAL in my catalog's tables return an error like:[PARSE_SYNTAX_ERROR] Syntax error at or near 'hive_metastore'. SQLSTATE: 42601 (line 1, pos 21)Why is this not supported in the dasboards but the s...
Both DESCRIBE and DESCRIBE DETAIL are supported by databricks sqlBut DESCRIBE DETAIL is only supported for tables not for viewsError you are getting is due to the SQL statement you have used is not valid and hence cannot be executedPlease check your ...
I'm interested in using the API to list all dashboards, but the API shows legacy dashboards only. Are there any plans to fix this API call to show all dashboards?I'm using this API call: https://docs.databricks.com/api/workspace/dashboards/listAnd th...
I understand the API is valid for legacy dashboards, but I need to include all dashboards, legacy or not.,
Hello,Currently, AI/BI Dashboards do not support Maps. This is a very important visual for those of us with geocoded data. We're stuck using Legacy Dashboards for this. Any ETA? Is this on the roadmap somewhere?Thanks,Angel
The team is actively working on supporting Maps in AI/BI Dashboards. You can expect Marker Map to be available in a few weeks and Choropleth to come early next year.
Dear community,Im trying to setup a connection to get data from a catalog / underlying tables from that catalog and load it directly into MS Access 365 (latest version). Is it possible to do this? What would be the best practice? Thanks,Bram
Hey guys,See here for a solution to this issue. You can specify the catalog when you setup the odbc driver.ODBC data source to connect to a Databricks catalo... - Databricks Community - 96354
Getting the below error when reading the snowflake tables data .We are successfully able to connect the respective tables .cannot display the data and other transformations.SparkConnectGrpcException: (net.snowflake.client.jdbc.SnowflakeSQLException) ...
Hi @ms_221 ,The SparkConnectGrpcException you're encountering, specifically mentioning that the incoming request with a certain IP/Token is not allowed to access Snowflake, suggests an issue related to network policies or access control within Snowfl...
