Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
I want to create notebooks for creating and accessing lakehouse . But I want it to be created using the assistant. Do we have a step by step guide to do it?
Open the assistant and ask it to spin up a lakehouse notebook and it should drop in the code you can run right away to connect and explore. If you're thinking to download the notebook, make you download it in pdf format and rename it after download ...
How can you control whether a file is a Notebook or a Query? Long story short - can you do anything to make Databricks recognise a .dbquery.ipynb file as a query and open it in SQL Warehouse?I created a file in my git folder as a query. All good. ...
Hello @dylan19 Good day!!Thats very frustrating! well just to check Did you change the .dbquery.ipynb file to a .sql file in your Git repository. For example, extract the SQL content from the .ipynb file (a JSON format) and saved it as plain text in ...
We were working with Python scripts on serverless (Git folder). The following code worked when we were calling on custom modules in sitting in the python scripts (eg. preprocess data from preprocessing.py). We've moved to shared compute but can no lo...
It seems you are using a workspace path. To confirm are you using a dedicated compute with access provided to a group. You need to provide can mange access to workspace path to the group.
Hi community, As the title says, I have been checking the documentation, but I could not find clear references to my question.I am working in a project, and I need to change all the plots fonts to match a specific brand. All plots are generated with ...
Set plt.rcParams['font.family'] = 'YourBrandFontName' at the start and all your plots will follow that font automatically, most probably.
Hi!We're trying to use the choropleth visualisation in the new dashboard. This works well for all countries we tried except for district level of Finland.The supported districts are clearly stated in the CSV on the documentation page titled county-di...
Hi Matt, You are correct that today, the Choropleth maps are using sub-regions as defined in https://en.wikipedia.org/wiki/Sub-regions_of_Finland. We are currently getting these boundaries from Mapbox, which provides this boundary data. Definitel...
I noticed a change in date formatting behavior in my dashboards starting yesterday.Details:My Workspace Setting: Default date format is set to YYYY-MM-DD (confirmed in user settings)Issue: Any new date fields added to dashboards are defaulting to DD/...
Hi @eliles ,You can check dashboard settings. From there you can change locale configuration, i.e set number and date formatting standards based on geographic or cultural preferencesDashboard settings - Azure Databricks | Microsoft Learn
We are trying to migrate our data from Fabric One Lake to the unity catalogue. Has anyone had experience with this before? Any pointers/ things to be aware of would be appreciated.
Hello @SP_6721 and @nayan_wylde Thank you both for your replies, I appreciate the pointers. I'll definitely look into your suggestions of creating an intermediate step instead of migrating directly from Fabric to databricks.
It will describe about Data models and how to build them on Databricks Platform, especially will Data vault and Data Mesh.Data VaultWhat is Data vault?Data Vault is a modern data modeling technique designed for agile, scalable, and auditable enterpri...
I am embedding a dashboard in my react application using an iframe.https://www.databricks.com/blog/how-embed-aibi-dashboards-your-websites-and-applicationshttps://docs.databricks.com/en/dashboards/index.html#embed-a-dashboard1. How can i let my users...
Hi , Has anyone gotten this working? I am facing the same issue when trying to get the embedded URL working from an iframe in my React application. I think I have all the necessary permissions in place, but still, the Databricks sign-in page opens in...
Hello,We are currently having an issue writing tables to the hive metastore. I've tried to outline what we know/have tried so far, below:Known situation:We have a databricks environment but it does not currently use Unity Catalogue. We instead have a...
Here are some recommendations and tips/tricks: 1. Understanding the Architecture & Common Issues In legacy Databricks setups, the Hive Metastore is used to manage tables and their metadata. When writing via ODBC (using the Simba Spark driver) or ...
Hi All I'm trying to create a Databricks SQL Endpoint using Terraform with the following resource configuration:resource "databricks_sql_endpoint" "dataproduct_sql_endpoint" { provider = databricks.workspace name ...
@SP_6721just a quick heads-up, this is all sorted now. Turns out the compute (warehouse) was just missing a tag. Added it, and everything started working as expected! Thank u !!
Hi Team,I'm encountering the following error when running queries on system tables through a Databricks dashboard using a Serverless SQL Warehouse:[QUERY_RESULT_WRITE_TO_CLOUD_STORE_FAILED] An internal error occurred while uploading the result set to...
Hello @Sitharth Good dayI think the cause would be this: By default, containers are missing from the databricks managed storage accountResolution: you need to create new workspace With this we can close the case.
Does anyone know when serverless SQL will be available for ap-southeast-1 region?
Serverless SQL is available now in this region
When trying incorporate an R package into my Spark workflow using the spark_apply() funciton in Sparklyr, I get the error:Error: java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.encoders.RowEncoder$.apply(Lorg/apache/spark/sql/types/StructT...
Hi @omjohn Can you try downgrading to a Databricks Runtime 13.3 LTS, which uses Spark 3.4.x, officially supported by sparklyr 1.8.1. I believe it would provide a more stable and better-tested integration.
I have the need to replicate existing unity catalog and workspaces from one region to another in databricks. Need help in understanding the different ways in which it can be done
Hey @DM3910 ,It may not be exactly what you need but there is some information in this article around cross-region/platform sharing:https://docs.databricks.com/aws/en/data-governance/unity-catalog/best-practices#cross-region-and-cross-platform-sharin...
