Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
Hi All,I've been using notebooks to run patch requests to an external API using the Python requests library. Often times certain notebooks will randomly start to fail throughout the day and will raise a ImportHookFinder.find_spec() error when attempt...
Hey, Did you manage to solve this issue? I am experiencing the same error but under a different context of using pytest in a notebook.My scenario:import pytest import os import sys os.chdir(f'/Workspace/Shared/DataTechnology-Databricks-Core') # Skip...
Photon ran out of memory while executing this query. Photon failed to reserve 32.0 KiB for BufferPool, in BroadcastBufferedRelation(spark_plan_id=13407). Memory usage: Total task memory (including non-Photon): 1829.7 MiB BroadcastBufferedRelation(spa...
Hello @vijamit Good day!!It was very hard to analyise the error but : The causes for the error shared are : Your query has run out of memory during execution, specifically when using the BuildHashedRelation and PartitionedRelation functions.Running o...
Using the old SQL editor, there was an option to add tags to saved queries under the "edit query info" option shown in the screenshots. Since starting to use the new editor, I noticed that option has disappeared from its previous location and I haven...
Hello @jtbing!At the moment, adding or editing tags for saved queries isn’t available in the new SQL editor UI. There’s also an upcoming Bricktalks session with the Product Manager for Databricks SQL, introducing the new SQL editor; you may find it u...
Hi All,I am getting below message when hovering on the Create SQL warehouse. I can only see the Serverless Starter Warehouse compute.You have reached the maximum number of SQL warehouses. Delete an existing warehouse to create new one.Regards,RN
I want to create notebooks for creating and accessing lakehouse . But I want it to be created using the assistant. Do we have a step by step guide to do it?
Open the assistant and ask it to spin up a lakehouse notebook and it should drop in the code you can run right away to connect and explore. If you're thinking to download the notebook, make you download it in pdf format and rename it after download ...
How can you control whether a file is a Notebook or a Query? Long story short - can you do anything to make Databricks recognise a .dbquery.ipynb file as a query and open it in SQL Warehouse?I created a file in my git folder as a query. All good. ...
Hello @dylan19 Good day!!Thats very frustrating! well just to check Did you change the .dbquery.ipynb file to a .sql file in your Git repository. For example, extract the SQL content from the .ipynb file (a JSON format) and saved it as plain text in ...
We were working with Python scripts on serverless (Git folder). The following code worked when we were calling on custom modules in sitting in the python scripts (eg. preprocess data from preprocessing.py). We've moved to shared compute but can no lo...
It seems you are using a workspace path. To confirm are you using a dedicated compute with access provided to a group. You need to provide can mange access to workspace path to the group.
Hi community, As the title says, I have been checking the documentation, but I could not find clear references to my question.I am working in a project, and I need to change all the plots fonts to match a specific brand. All plots are generated with ...
Set plt.rcParams['font.family'] = 'YourBrandFontName' at the start and all your plots will follow that font automatically, most probably.
In a Databricks AI/BI dashboard, I have a field with multiple categories (e.g., district-wise sales with 50 districts). How can I display only the top few categories (like the top 10) based on a specific metric such as sales?
@GunaR is this not where we'd want to leverage a parameter? And then pair this with a filter on the dashboard? Perhaps you create an aggregate SQL query for the various metrics If you can't use a parameter in the ORDER BY clause, which you likely can...
Hi!We're trying to use the choropleth visualisation in the new dashboard. This works well for all countries we tried except for district level of Finland.The supported districts are clearly stated in the CSV on the documentation page titled county-di...
Hi Matt, You are correct that today, the Choropleth maps are using sub-regions as defined in https://en.wikipedia.org/wiki/Sub-regions_of_Finland. We are currently getting these boundaries from Mapbox, which provides this boundary data. Definitel...
I noticed a change in date formatting behavior in my dashboards starting yesterday.Details:My Workspace Setting: Default date format is set to YYYY-MM-DD (confirmed in user settings)Issue: Any new date fields added to dashboards are defaulting to DD/...
Hi @eliles ,You can check dashboard settings. From there you can change locale configuration, i.e set number and date formatting standards based on geographic or cultural preferencesDashboard settings - Azure Databricks | Microsoft Learn
We are trying to migrate our data from Fabric One Lake to the unity catalogue. Has anyone had experience with this before? Any pointers/ things to be aware of would be appreciated.
Hello @SP_6721 and @nayan_wylde Thank you both for your replies, I appreciate the pointers. I'll definitely look into your suggestions of creating an intermediate step instead of migrating directly from Fabric to databricks.
It will describe about Data models and how to build them on Databricks Platform, especially will Data vault and Data Mesh.Data VaultWhat is Data vault?Data Vault is a modern data modeling technique designed for agile, scalable, and auditable enterpri...
I am embedding a dashboard in my react application using an iframe.https://www.databricks.com/blog/how-embed-aibi-dashboards-your-websites-and-applicationshttps://docs.databricks.com/en/dashboards/index.html#embed-a-dashboard1. How can i let my users...
Hi , Has anyone gotten this working? I am facing the same issue when trying to get the embedded URL working from an iframe in my React application. I think I have all the necessary permissions in place, but still, the Databricks sign-in page opens in...
Hello,We are currently having an issue writing tables to the hive metastore. I've tried to outline what we know/have tried so far, below:Known situation:We have a databricks environment but it does not currently use Unity Catalogue. We instead have a...
Here are some recommendations and tips/tricks: 1. Understanding the Architecture & Common Issues In legacy Databricks setups, the Hive Metastore is used to manage tables and their metadata. When writing via ODBC (using the Simba Spark driver) or ...
