- 54 Views
- 1 replies
- 1 kudos
Regression: Dashboard slicer paste now commits invalid filter values instead of searching
Hi Team,We appear to be experiencing a recent regression in the AI/BI dashboard filter slicer behaviour.Steps to reproduceOpen a dashboard containing a single-select or multi-select filter slicer.Click into the slicer’s text input.Paste text from the...
- 54 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi Tim, I can't find any mention of this internally. But I suspect it will be related to this change Multi-select filter paste: Viewers can now copy a column of values from a spreadsheet and paste them into a multi-select filter. My recommendation w...
- 1 kudos
- 7 Views
- 0 replies
- 0 kudos
Vacuum on DLT
We are currently using DLT tables in our target tables. The tables are getting loaded in continuous job pipelines.The liquid cluster is enabled in the tables. Will Vacuum work on these tables when it is getting loaded in continuous mode? How to run t...
- 7 Views
- 0 replies
- 0 kudos
- 61 Views
- 3 replies
- 0 kudos
Data Ingestions errors
I was going to ingestion Data from website to databricks but it is showing Public DBFS is not enableb message. is there any other way to automate data ingestion to databricks?
- 61 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @sher_1222 yes you can upload to cloud storage and then connect using unity catalog: Connect to cloud object storage using Unity Catalog - Azure Databricks | Microsoft Learnand then use What is Auto Loader? | Databricks on AWS to automatically ing...
- 0 kudos
- 106 Views
- 3 replies
- 1 kudos
Azure Databricks UI consuming way too much memory & laggy
This especially happens when the notebook is large with many cells. Even if I clear all the outputs scrolling the notebook is way too laggy. When I start running the code the memory consumption is 3-4GB minimum even if I am not displaying any data/ta...
- 106 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi, these are teh recommended troubleshooting steps we have: Troubleshooting & Immediate Workarounds Browser Recommendations: Use an incognito/private window to avoid interference from browser extensions/ad blockers.Monitor memory consumption; close...
- 1 kudos
- 82 Views
- 3 replies
- 0 kudos
Serverless notebook DNS failure (gai error / name resolution)
I’m using a Databricks workspace on AWS (region: us-west-2). My Serverless notebook (CPU) cannot access any external URL — every outbound request fails at DNS resolution.Minimal test in a notebook:import urllib.requesturllib.request.urlopen("https://...
- 82 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi, Here are some troubleshooting steps: 1. Network Connectivity Configuration (NCC) Confirm that the correct NCC (such as ncc_public_internet) is attached specifically to Serverless compute, not just to SQL Warehouses or other resources.After making...
- 0 kudos
- 43529 Views
- 8 replies
- 5 kudos
Python mocking dbutils in unittests
I am trying to write some unittests using pytest, but I am coming accross the problem of how to mock my dbutils method when dbutils isn't being defined in my notebook.Is there a way to do this so that I can unit test individual functions that are uti...
- 43529 Views
- 8 replies
- 5 kudos
- 5 kudos
If this helps anyone here is how we do this:We rely on databricks_test for injecting dbutils into the notebooks that we're testing (which is a 3rd party package mind you and hasn't been updated in a while but still works). And in our notebooks we put...
- 5 kudos
- 85 Views
- 4 replies
- 2 kudos
st_point is disabled or unsupported.
On my DLT pipeline,I Installed the Databricks-mosaic libraryPhoton is activatedI'm using a workspace premium tierSELECT id, city_name, st_point(latitude, longitude) AS city_point FROM city_data ;st_point is disabled or unsupported. Co...
- 85 Views
- 4 replies
- 2 kudos
- 2 kudos
DLT pipelines are still on runtime 16.4 which doesn't have support for st_point yet. See details here https://learn.microsoft.com/en-us/azure/databricks/release-notes/dlt/ You should be able to use st_point in normal SQL editor as long as the cluster...
- 2 kudos
- 123 Views
- 1 replies
- 0 kudos
Declarative Pipelines - Dynamic Overwrite
Regarding the limitations of declarative pipelines—specifically the inability to use replaceWhere—I discovered through testing that materialized views actually support dynamic overwrites. This handles several scenarios where replaceWhere would typica...
- 123 Views
- 1 replies
- 0 kudos
- 0 kudos
This is a really interesting find, and honestly not something most people expect from materialized views.Under the hood, MVs in Databricks declarative pipelines are still Delta tables. So when you set partitionOverwriteMode=dynamic and partition by a...
- 0 kudos
- 56 Views
- 2 replies
- 0 kudos
power BI Vnet data gateway to Databricks using import mode
we are using Power Bi Vnet data gateway and data source connection as databricks and using import mode.databricks is behind Vnet.refreshing model working fine for 400 records but larger volume throwing errors.i tried with different way kind of increm...
- 56 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @szymon_dybczak thanks but that is what we set when we do make power Bi desktop model ,i used this query only and made semantic model in power BI desktop and then we did Publish this into power bi service and do the refresh in web UI there it is f...
- 0 kudos
- 911 Views
- 7 replies
- 7 kudos
Resolved! List budget policies applying filter_by
I'm trying to list budget policies using the parameter "filter_by" to filter policies that start with "aaaa" but I'm getting an error "400 Bad Request"{'error_code': 'MALFORMED_REQUEST', 'message': "Could not parse request object: Expected 'START_OB...
- 911 Views
- 7 replies
- 7 kudos
- 7 kudos
Thanks for the reply, @szymon_dybczak and @lingareddy_Alva.I tried both approaches but none was successful.url = f'{account_url}/api/2.1/accounts/{account_id}/budget-policies'filter_by_json = json.dumps({"policy_name": "aaaa"})params = {"filter_by": ...
- 7 kudos
- 1400 Views
- 4 replies
- 0 kudos
Using Lakehouse Federation for SQL Server with Serverless Compute
Hi,My team was able to create a Foreign Catalog that connects to a SQL Server instance hosted on an Azure VM, however when trying to query the catalog, we cannot access it using serverless compute (or a serverless sql warehouse). We have tried lookin...
- 1400 Views
- 4 replies
- 0 kudos
- 0 kudos
I'm trying to get something similar to work: Lakehouse Federation for Oracle with SQL warehouse serverless. We are using Azure Databricks and our Oracle DB runs on-prem. I've been able to use classic compute to query the database, but now I'd like to...
- 0 kudos
- 1806 Views
- 3 replies
- 4 kudos
Resolved! Why Does Azure Databricks Consume So Much Memory When Running in the Background?
I had two Azure Databricks pages open in my browser without performing any computations. When I returned from lunch, I noticed that they were occupying about 80% of the memory in the task manager. What happened? This issue never occurred in the past,...
- 1806 Views
- 3 replies
- 4 kudos
- 4 kudos
@szymon_dybczak This issue still exists and is getting worse. Even a 32GB memory & ultra 7 processor laptop cannot seem to handle this issue if there are many cells in the notebook. Do you know when it'll be fixed?
- 4 kudos
- 104 Views
- 3 replies
- 0 kudos
workspace notebook path not recognized by dbutils.notebook.run() when running from a workflow/job
result = dbutils.notebooks.run("/Workspace/YourFolder/NotebookA", timeout_seconds=600, arguments={"param1": "value1"}) print(result)I was able to execute the above code manually from a notebook.But when i run the same notebook as a job, it fails stat...
- 104 Views
- 3 replies
- 0 kudos
- 0 kudos
@siva_pusarla , Try to convert env_setup into repo-based code and control behavior via environmentInstead of a workspace notebook, use a Python module in the repo and drive environment differences using:Job parametersBranches (dev / test / prod)Secre...
- 0 kudos
- 480 Views
- 6 replies
- 3 kudos
Read Array of Arrays of Objects JSON file using Spark
Hi Databricks Community! This is my first post in this forum, so I hope you can forgive me if it's not according to the forum best practices After lots of searching, I decided to share the peculiar issue I'm running into in this community.I try to lo...
- 480 Views
- 6 replies
- 3 kudos
- 3 kudos
I guess I was a bit over enthusiastic by accepting the answer.When I run the following on the single object array of arrays (as shown in the original post) I get a single row with column "value" and value null. from pyspark.sql import functions as F,...
- 3 kudos
- 76 Views
- 1 replies
- 1 kudos
Azure databricks streamlit app unity catalog access
Hi allI am developing a Databricks app. I will use Databricks asset bundles for deployment.How can I connect Databricks streamlit app into Databricks unity catalog?Where should I define the credentials? (Databricks host for dev, qa and prod environme...
- 76 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, As a starter you may want to try deploying the streamlit starter app from the app UI, this will show you the pattern to connect and pull data into your streamlit app. The following then gives some best practise guidelines on your questions: 1. U...
- 1 kudos
| User | Count |
|---|---|
| 1632 | |
| 791 | |
| 517 | |
| 349 | |
| 287 |
