- 918 Views
- 1 replies
- 1 kudos
Resolved! Databricks Dashboard Issue: No Mouse-Based Navigation When Dashboard Tabs Exceed the Top Ribbon
When dashboards have many pages, the top tab bar overflows and can’t be navigated using the mouse. Only left keyboard arrow, right keyboard arrow works, which is slow and inconvenient not user friendly.Expected: ability to scroll tabs with mouse e.g....
- 918 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @surajitDE! You can use the horizontal scroll bar to navigate through the dashboard pages.If you’re on a trackpad, you can simply scroll horizontally. If you’re using a mouse, you can: Hold Shift and scroll with the mouse wheel (easiest), orDra...
- 1 kudos
- 7510 Views
- 4 replies
- 6 kudos
Resolved! How to use variable-overrides.json for environment-specific configuration in Asset Bundles?
Hi all,Could someone clarify the intended usage of the variable-overrides.json file in Databricks Asset Bundles?Let me give some context. Let's say my repository layout looks like this:databricks/ ├── notebooks/ │ └── notebook.ipynb ├── resources/ ...
- 7510 Views
- 4 replies
- 6 kudos
- 6 kudos
It does. Thanks for the reponse. I also continued playing around with it and found a way using the variable-overrides.json file. I'll leave it here just in case anyone is interested:Repository layout:databricks/ ├── notebooks/ │ └── notebook.ipynb ...
- 6 kudos
- 4222 Views
- 4 replies
- 1 kudos
Best practices for optimizing Spark jobs
What are some best practices for optimizing Spark jobs in Databricks, especially when dealing large datasets? Any tips or resources would be greatly appreciated! I’m trying to analyze data on restaurant menu prices so that insights would be especiall...
- 4222 Views
- 4 replies
- 1 kudos
- 1 kudos
In addition to above cool comments, try to use clusters with VMs enabled for disk caching as well. This caches data at parquet files level in VM local storage, acting as a great complement to spark caching.
- 1 kudos
- 768 Views
- 0 replies
- 1 kudos
Agent Bricks Webinar
Our Databricks x Unifeye Meetup community just hit 150 members! A huge milestone, especially considering we’ve consistently had 50+ people joining every webinar. The momentum is real, and the audience keeps growing! This week, we’re taking it one s...
- 768 Views
- 0 replies
- 1 kudos
- 3608 Views
- 2 replies
- 1 kudos
Resolved! SQL cell v spark.sql in notebooks
I am fairly new to Databricks, and indeed Python, so apologies if this has been answered elsewhere but I've been unable to find it.I have been mainly working in notebooks as opposed to the SQL editor, but coding in SQL where possible using SQL cells ...
- 3608 Views
- 2 replies
- 1 kudos
- 9913 Views
- 5 replies
- 3 kudos
Connect to Onelake using Service Principal, Unity Catalog and Databricks Access Connector
We are trying to connect Databricks to OneLake, to read data from a Fabric workspace into Databricks, using a notebook. We also use Unity Catalog. We are able to read data from the workspace with a Service Principal like this:from pyspark.sql.types i...
- 9913 Views
- 5 replies
- 3 kudos
- 3 kudos
As commented you need to assign "Storage Blob Data Contributor or Storage Account Contributor to the service principal you're using in the "connection" provided to the "external location". Another more advanced and even better option would be to use ...
- 3 kudos
- 5488 Views
- 1 replies
- 0 kudos
Updating a Delta Table in Delta Live Tables (DLT) from Two Event Hubs
I am working with Databricks Delta Live Tables (DLT) and need to ingest data from two different Event Hubs. My goal is to:Ingest initial data from the first Event Hub (Predictor) and store it in a Delta Table (data_predictions).Later, update this tab...
- 5488 Views
- 1 replies
- 0 kudos
- 0 kudos
To achieve robust, persistent CDC (Change Data Capture)–style updates in Databricks DLT with your scenario—while keeping data_predictions as a Delta Table (not a Materialized View)—you need to carefully avoid streaming joins and side effects across s...
- 0 kudos
- 4486 Views
- 1 replies
- 1 kudos
Resolved! Databricks UMF Best Practice
Hi there, I would like to get some feedback on what are the ideal/suggested ways to get UMF data from our Azure cloud into Databricks. For context, UMF can mean either:User Managed FileUser Maintained FileBasically, a UMF could be something like a si...
- 4486 Views
- 1 replies
- 1 kudos
- 1 kudos
Several effective patterns exist for ingesting User Managed Files (UMF) such as CSVs from Azure into Databricks, each with different trade-offs depending on governance, user interface preferences, and integration with Microsoft 365 services. Common A...
- 1 kudos
- 5223 Views
- 1 replies
- 0 kudos
DLT detecting changes but not applying them
We have three source tables used for a streaming dimension table in silver. Around 50K records are changed in one of the source tables, and the DLT pipeline shows that it has updated those 50K records, but they remain unchanged. The only way to pick ...
- 5223 Views
- 1 replies
- 0 kudos
- 0 kudos
The most likely reason your DLT pipeline shows 50K updates but the records remain unchanged is related to how Delta Live Tables (DLT) handle streaming tables, update logic, and schema constraints. When the target table uses an auto-increment ID (espe...
- 0 kudos
- 4174 Views
- 1 replies
- 0 kudos
Issue with Disabled "Repair DAG", "Repair All DAGs" Buttons in Airflow UI, functionality is working.
We are encountering an issue in the Airflow UI where the 'Repair DAG' and 'Repair All DAGs' options are disabled when a specific task fails. While the repair functionality itself is working properly (i.e., the DAGs can still be repaired through execu...
- 4174 Views
- 1 replies
- 0 kudos
- 0 kudos
The issue with the 'Repair DAG' and 'Repair All DAGs' options being disabled in the Airflow UI when using the Databricks Workflow Operator is a known UI-specific problem that does not affect backend execution or the actual repair functionality. While...
- 0 kudos
- 7022 Views
- 1 replies
- 1 kudos
How to Fetch Azure OpenAI api_version and engine Dynamically After Resource Creation via Python?
Hello,I am using Python to automate the creation of Azure OpenAI resources via the Azure Management API. I am successfully able to create the resource, but I need to dynamically fetch the following details after the resource is created:API Version (a...
- 7022 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi Sudheer, It's been a while since you posted, but are you still facing this issue? Here are a few things you could check if needed: API version In Azure OpenAI, api-version is a query parameter on the data-plane (inference) requests, not a proper...
- 1 kudos
- 2515 Views
- 5 replies
- 2 kudos
Resolved! #data bricks snowflake dialect
Hello,I’m encountering an issue while converting SQL code to the Lake bridge Snowflake dialect. It seems that DML and DDL statements may not be supported in the Snowflake dialect within Lake bridge.Could you please confirm whether DML and DDL stateme...
- 2515 Views
- 5 replies
- 2 kudos
- 2 kudos
@jadhav_vikas , I did some digging through internal docs and I have some hints/suggestions. Short answer Databricks Lakehouse Federation (often referred to as “Lakehouse Bridge”) provides read‑only access to Snowflake; DML and DDL are not supported ...
- 2 kudos
- 1367 Views
- 3 replies
- 0 kudos
Resolved! API call fails to initiate create Service Principal secret
Hi,I've constructed an AWS lambda function which is used to auto rotate my Service Principal Secret in the Databricks account. Authentication is setup with OAuth2, the api call for the token generation is successful but when executing the api call to...
- 1367 Views
- 3 replies
- 0 kudos
- 0 kudos
Your error message, "Invalid service principal id," typically indicates a mismatch or formatting problem with the service principal's unique identifier in your API request. Although you checked the client_id, this value is not always the one needed f...
- 0 kudos
- 1417 Views
- 1 replies
- 2 kudos
Databricks Taiwan User Community
Would be great have a group for databricks Taiwan community.
- 1417 Views
- 1 replies
- 2 kudos
- 2 kudos
Hey @kennyhsieh , hope it's not too late to reply to the post! A "Databricks User Group Taiwan" has recently been formed on LinkedIn.
- 2 kudos
- 4686 Views
- 2 replies
- 0 kudos
Can AWS workspaces share subnets?
The docs state:"You can choose to share one subnet across multiple workspaces or both subnets across workspaces."as well as:"You can reuse existing security groups rather than create new ones."and on this page:"If you plan to share a VPC and subnets ...
- 4686 Views
- 2 replies
- 0 kudos
- 0 kudos
AWS WorkSpaces can be configured with subnets that can be shared within an AWS account or across AWS accounts using resource sharing mechanisms, but this depends on the specific AWS service and context. For Databricks workspaces on AWS, documentation...
- 0 kudos
