Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
Hello,We recently modified some table names and used the UC Lineage to identify the users that were accessing these particular tables to notify. However, it seems we broke our Genie Space that was using that table. As best as I can tell, queries from...
As of now, system tables like information_schema.table_usage or query_history do not log Genie Space queries. They primarily log interactive and scheduled queries from notebooks, dashboards, and the SQL editor.you will need to rely on Genie configura...
Bug SummaryTitle: Spark SQL internal error when executing SQL code in the Azure Databricks SQL Editor DescriptionWhen executing SQL code in the Azure Databricks SQL Editor, a Spark SQL internal error is returned. Steps to ReproduceCopy and paste th...
Check Databricks runtime version:If possible, upgrade to the latest Databricks runtime, as internal errors are sometimes fixed in newer versions.
Hi!I try to connect to an azure databricks sql warehouse in dbeaver, which uses the azure databricks jdbc driver version 2.7.1 ## and I cannot get M2M authentication to work. I get a 'Not Authorized' (401) response when I try to connect, and it seems...
Hey @jakobhaggstrom,It's possible to connect to DBX using an Azure service principal. You're just missing the OIDC URL in the JDBC driver's parameters.The OIDC discovery is enabled by default but you need to set its URL which contains your Azure tena...
Hi -- I'm trying to connect to BigQuery as a foreign catalog. I'm able to create and successfully test the connection, but when I create a foreign catalog it appears empty, and queries against that catalog return a "TABLE_OR_VIEW_NOT_FOUND" error.The...
Thanks @lingareddy_Alva! This is helpful.I reached out to ask about the required permissions, we'll see how that goes.In the meantime, I'm trying to test out the other two paths. Do you know where I could find docs or examples for those operations? I...
Hi everyone,I recently started using AI/BI dashboard and I'm struggling with this glitch in the map. The visual was generated without any issues but the cross-filtering of the map for example, if I click on a county, the rest of the chart values chan...
Hey @meljung ,The issue seems to stem from using an aggregated metric (`count_distinct(user_id)`) in the map visual.When clicking on a county, the map attempts to filter other visuals using the underlying dimension, but inconsistencies may occur if t...
Hi, I'm Shun from Japan.I tried to create Databricks Dashboard using Asset Bundles.But, I got an error message bellow on Web Terminal.$ databricks bundle deployWarn: Failed to load git info from /api/2.0/workspace/get-statusError: failed to get dashb...
Hi @Shun_T ,The error message you're seeing,usually means that the dashboard you're trying to deploy either doesn’t exist in your Databricks workspace,or the ID/reference used in the YAML file is invalid.Here are a few steps that should help:1.Verify...
I do no like the fact that 'all workspace users' can manage' the dashboard
Hey @source2sea ,I ran into the same issue before. The reason it shows "All workspace users – Can Manage (inherited)" is because the permission is coming from the folder where your dashboard is saved.To fix it, go to that folder, click on the three d...
Hello,I am struggling trying to assign permission dynamically on a schema. I am using databricks asset bundle and I have a parametrized script to assign permission.grant select on schema {{schema}} to {{group}};I cannot achieve a dynamic grant statem...
I use notebooks as part of the asset bundle deployment to conduct a lot of dynamic configurations based upon the workspace being deployed to (ex. Development, Test, Production). In conjunction, I developed a helper Python library with a number of fu...
Is it possible it use Databricks Free Edition + dbt cloud for dbt projects?
Hi @josh__, I guess yes, you can use Databricks Free Edition with dbt Cloud for dbt projects. Just keep in mind that the Free Edition is mainly meant for learning and experimentation, so it does have some limitations around features and resources. Mi...
Hi! I have these two tables which are altered(renamed) views from my main table, and there seems to be no way of joining this tables. Everytime I try Databricks indicates that these tables cannot be found, but if I go and write a Select * statements ...
Hi @ArturSvoboda This sounds like a classic Databricks workspace isolation issue. Here are the most likely causes and things to check:1. Temporary Views vs Persistent ViewsYour renamed views might be temporary views that only exist in your current Sp...
Hi,I'm new to databricks, and I'm trying to create an area plot in a visualisation, the only problem is that I do not see the 'group' otion in the y axis section. I see 'Color' but that is all, not 'group by' option (I'm expecting to see 'Color/Grou...
Hi @BryanM Use "Color by Y-Series" for GroupingIn Databricks, the "Color by Y-Series" section IS your grouping mechanism for area charts:Add your grouping dimension (the field you want to group by) to the "Color by Y-Series" sectionThis will create s...
Hi,I am trying to use pm4py library to visualize my data. my code executes perfectly but there is no visualization how can i solve this problem ? There is no error message#processmining # databricks #pm4py
I believe we do not support GUI-based viewers like pm4py.visualizer.view(gviz), which tries to open a file using your system's default image/PDF viewer.Could you try visualizing and then saving the output - import pm4py# Sample log (replace with your...
Here, we're trying to use the Python UDF inside the query.taking the table as function input converting the table into dataframe performing modification converting the dataframe into table returning the table How can we create spark context inside U...
Hi team,I believe you cannot create or access a SparkSession or run Spark operations like spark.sql() directly inside a Python UDF. input_table is a table argument, not a string with a table name. You receive it as a pandas DataFrame when using RETUR...
Hi,Is it possible to connect to the Community Edition using JDBC?On the web dashboard, in the "compute" instance parameters, there are details about a JDBC connection string, but it is templated to use a PAT. Though I've read in other posts that PAT ...
Torch3333, keep an eye out next week for announcments at DAIS26. I can't reveal anything but you question will have a definite answer. Cheers, Lou.
Hello!I'm new to Databricks.Assume, I need to migrate 2 Tb Oracle Datamart to Databricks on Azure. Serverless SQL Warehouse seems as a valid choice.What is a better option ( cost vs performance) to store the data?Should I upload Oracle Extracts to Az...
@Curious-mind You got it. Running the COPY INTO is good for the initial load as it's optimized for bulk loads. You'll want to use Auto-loader going forward to incrementally process new rows.
