Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
When trying incorporate an R package into my Spark workflow using the spark_apply() funciton in Sparklyr, I get the error:Error: java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.encoders.RowEncoder$.apply(Lorg/apache/spark/sql/types/StructT...
Hi @omjohn Can you try downgrading to a Databricks Runtime 13.3 LTS, which uses Spark 3.4.x, officially supported by sparklyr 1.8.1. I believe it would provide a more stable and better-tested integration.
I have the need to replicate existing unity catalog and workspaces from one region to another in databricks. Need help in understanding the different ways in which it can be done
Hey @DM3910 ,It may not be exactly what you need but there is some information in this article around cross-region/platform sharing:https://docs.databricks.com/aws/en/data-governance/unity-catalog/best-practices#cross-region-and-cross-platform-sharin...
Hello,I'm fairly new to databricks, and wanted to play around with creating dashboards similar to ones I've done in Tableau. With the genie being limited at times in graphs it can create, I used the SQL editor to get what I wanted; however, I'm havin...
Hello @Shivla! This issue might be related to how the region fields are configured in the choropleth map. If the same State column is assigned to both the Country and State/Province fields, the map may not load properly. Try assigning the state colum...
It is disappointing to see that Databricks is quite at a premature stage and has just marketed the product, but the features are not ready to use yet.Our requirement is fairly straightforward: to create an AI/BI dashboard in Databricks, publish and e...
Hi @Amit-Kumar Have you considered consumer access? If your users are internal users, you can onboard them as workspace users and provision a read-only access to dashboards and Genie. Alternatively, you could use embedded credentials approach for acc...
The data selection is part of loaded data if I want to filter out the data I'm not able to filter it out I believe this is glitch in the data bricks dashboard UI
Attempting to create a Python UDF and receiving an error stating it's not supported in my environment.running on SQL Serverless Cluster - Preview (v 2025.16)[FEATURE UNAVAILABLE]Python UDTF is not supported in your environment. To use this feature, p...
Currently Serverless supports only scaler valued function not table valued function. also you need to provide the catalog and schema in the function.You can create using a pro computecreate function {catalog_name}.{schema_name}._rrr_taxonomy_test()RE...
According to the pricing page, the minimum cost in using this product is 1 DBU per hour.Often times, access to such tables comes in bursts with many periods of idle time. How is infrequent use factored into the pricing? This is also important for set...
As far as I can tell, the minimum charge for an online table is DBU-cost x 24 x 30 per month, i.e. around $720 per month.Is the 15-minute scale-down window documented somewhere?With a 15-minute scale-down window, assuming just one access per hour, th...
Hello,We recently modified some table names and used the UC Lineage to identify the users that were accessing these particular tables to notify. However, it seems we broke our Genie Space that was using that table. As best as I can tell, queries from...
As of now, system tables like information_schema.table_usage or query_history do not log Genie Space queries. They primarily log interactive and scheduled queries from notebooks, dashboards, and the SQL editor.you will need to rely on Genie configura...
Bug SummaryTitle: Spark SQL internal error when executing SQL code in the Azure Databricks SQL Editor DescriptionWhen executing SQL code in the Azure Databricks SQL Editor, a Spark SQL internal error is returned. Steps to ReproduceCopy and paste th...
Check Databricks runtime version:If possible, upgrade to the latest Databricks runtime, as internal errors are sometimes fixed in newer versions.
Hi!I try to connect to an azure databricks sql warehouse in dbeaver, which uses the azure databricks jdbc driver version 2.7.1 ## and I cannot get M2M authentication to work. I get a 'Not Authorized' (401) response when I try to connect, and it seems...
Hey @jakobhaggstrom,It's possible to connect to DBX using an Azure service principal. You're just missing the OIDC URL in the JDBC driver's parameters.The OIDC discovery is enabled by default but you need to set its URL which contains your Azure tena...
Hi -- I'm trying to connect to BigQuery as a foreign catalog. I'm able to create and successfully test the connection, but when I create a foreign catalog it appears empty, and queries against that catalog return a "TABLE_OR_VIEW_NOT_FOUND" error.The...
Thanks @lingareddy_Alva! This is helpful.I reached out to ask about the required permissions, we'll see how that goes.In the meantime, I'm trying to test out the other two paths. Do you know where I could find docs or examples for those operations? I...
Hi everyone,I recently started using AI/BI dashboard and I'm struggling with this glitch in the map. The visual was generated without any issues but the cross-filtering of the map for example, if I click on a county, the rest of the chart values chan...
Hey @meljung ,The issue seems to stem from using an aggregated metric (`count_distinct(user_id)`) in the map visual.When clicking on a county, the map attempts to filter other visuals using the underlying dimension, but inconsistencies may occur if t...
Hi, I'm Shun from Japan.I tried to create Databricks Dashboard using Asset Bundles.But, I got an error message bellow on Web Terminal.$ databricks bundle deployWarn: Failed to load git info from /api/2.0/workspace/get-statusError: failed to get dashb...
Hi @Shun_T ,The error message you're seeing,usually means that the dashboard you're trying to deploy either doesn’t exist in your Databricks workspace,or the ID/reference used in the YAML file is invalid.Here are a few steps that should help:1.Verify...
I do no like the fact that 'all workspace users' can manage' the dashboard
Hey @source2sea ,I ran into the same issue before. The reason it shows "All workspace users – Can Manage (inherited)" is because the permission is coming from the folder where your dashboard is saved.To fix it, go to that folder, click on the three d...
Hello,I am struggling trying to assign permission dynamically on a schema. I am using databricks asset bundle and I have a parametrized script to assign permission.grant select on schema {{schema}} to {{group}};I cannot achieve a dynamic grant statem...
I use notebooks as part of the asset bundle deployment to conduct a lot of dynamic configurations based upon the workspace being deployed to (ex. Development, Test, Production). In conjunction, I developed a helper Python library with a number of fu...
