- 50 Views
- 2 replies
- 0 kudos
Serverless compute cannot access internet
I'm experiencing "Network is unreachable" errors when trying to access external APIs from serverless compute jobs, despite having a network policy configured to allow access to all destinations. I have the "Serverless egress control" in the account s...
- 50 Views
- 2 replies
- 0 kudos
- 0 kudos
The ongoing "Network is unreachable" errors in Databricks serverless compute suggest that network egress is being blocked, even though you have "Full access" set for Serverless Egress Control and a permissive network policy. The partial DNS resolutio...
- 0 kudos
- 9 Views
- 0 replies
- 0 kudos
disabling a task in a databricks job
Hello I have jobs that perform a number of activities.Sometimes I want to disable one or more of these activities.I can do that easily in an app like ADF.Reading around I cannot find an easy way to do this although, what I've read suggest that it was...
- 9 Views
- 0 replies
- 0 kudos
- 3256 Views
- 6 replies
- 2 kudos
Resolved! Trouble Enabling File Events For An External Location
Hello all,I am trying to enable file events on my Azure Workspace for the File Arrival Trigger trigger mode for Databricks Workflows. I'm following this documentation exactly (I think) but I'm not seeing the option to enable them. As you can see here...
- 3256 Views
- 6 replies
- 2 kudos
- 2 kudos
I also have an external location that is a ADLS v2 however from databricks I only want to grant read access to the managed identity. I want to use file arrival trigger but want to have option of using more than 50 such triggers. Is it possible to onl...
- 2 kudos
- 5474 Views
- 12 replies
- 1 kudos
Resolved! Options to access files in the community edition
As of now DBFS option is disabled in the Databricks community edition. What are the other ways to use file in the Databricks notebooks for learning? When I go to catalog it show default option only with the AWS S3. Is it the only option to access the...
- 5474 Views
- 12 replies
- 1 kudos
- 34 Views
- 1 replies
- 2 kudos
Databricks Architecture Center
Databricks Architecture Center — Your Blueprint for Building Modern Data & AI PlatformsThe Databricks Architecture Center is a centralized knowledge hub that provides:End-to-end reference architecturesIndustry-specific patternsArchitecture decision ...
- 34 Views
- 1 replies
- 2 kudos
- 2 kudos
Its very useful. I suppose you missed to provide the link. Here is the link for easy accesshttps://www.databricks.com/resources/architectures
- 2 kudos
- 40 Views
- 3 replies
- 0 kudos
How do I choose between a standard cluster and a shared cluster in Databricks?
When should I use a single-user cluster, and when should I use a multi-user/shared cluster? What’s the difference and how do I pick the right one?
- 40 Views
- 3 replies
- 0 kudos
- 0 kudos
Agree with previous responses BUT consider using a dedicated cluster if using it under an "automation security principal" identity for a subset of your jobs, if that makes sense. To summarize, you need to run a job, under a non-human identity and you...
- 0 kudos
- 11096 Views
- 13 replies
- 2 kudos
Install the ODBC Driver 17 for SQL Server
Hi,I am not a Data Engineer, I want to connect to ssas. It looks like it can be connected through pyodbc. however looks like I need to install "ODBC Driver 17 for SQL Server" using the following command. How do i install the driver on the cluster an...
- 11096 Views
- 13 replies
- 2 kudos
- 2 kudos
To connect to SSAS using pyodbc, you do need the "ODBC Driver 17 for SQL Server" installed on your system or cluster. On most Linux clusters, you can install it using Microsoft’s official instructions, which typically involve adding the Microsoft rep...
- 2 kudos
- 66 Views
- 3 replies
- 0 kudos
POC on spark 4.x
I need to do some POC with spark 3.5.7 and 4.x and need some local setup with some sample Kafka source. The POC would read data from Kafka via streaming job and write to delta table and I would like to do this on spark-4.x ..Do you know of any quick ...
- 66 Views
- 3 replies
- 0 kudos
- 0 kudos
Hello @RevanthV , I did some digging and here are some helpful tips. Got it — here are fast, reproducible ways to stand up a local Kafka source and run a Spark Structured Streaming job that writes to a Delta table, plus the common fixes for the conne...
- 0 kudos
- 620 Views
- 2 replies
- 1 kudos
Dashboard Filters (Showing Description) but selecting the id value for use query parameters
HiIs there a way to have a Single or multiple filter in a Dashboard that shows titles/names but on selection passes in the identifier as query parameter filters. The cost management dashboard shipped by Databricks seems to do this for the workspace s...
- 620 Views
- 2 replies
- 1 kudos
- 1 kudos
I wanted to share my solution to this, hope this helps. Create a SQL function that accepts an ARRAY<STRING>, extracts a numeric ID enclosed in parentheses from the end of each string, and returns an ARRAY<BIGINT>.Example UsageInput: ["Item (1)", "Ite...
- 1 kudos
- 146 Views
- 3 replies
- 2 kudos
Resolved! Cannot view nested MLflow experiment runs without changing URL
Hello,I've recently been testing out Databricks experiments for a project of mine. I wanted to nest runs, and then see these runs grouped by their parent in the experiments UI. For the longest time, I couldn't figure out how to do this. I was seeing ...
- 146 Views
- 3 replies
- 2 kudos
- 2 kudos
@africke , If you’re happy with the results, please go ahead and accept this as the solution so others know it worked.
- 2 kudos
- 89 Views
- 4 replies
- 1 kudos
Resolved! Got an empty query file when cloning Query file.
Hello Community,In our AWS Databricks environment, we’ve encountered some behavior we don’t understand while performing the following operation.When we clone a query file that already has content, a new file is created with the same name and “(clone)...
- 89 Views
- 4 replies
- 1 kudos
- 1 kudos
@kyeongmin_baek - There is no aut-save or CMD+S for Query as it get saved only once it is attached to a cluster AND 'SAVE' icon is used. However, it still stays in cache as unsaved in that 'query' window but cloning or other file operations may lose ...
- 1 kudos
- 79 Views
- 2 replies
- 0 kudos
Error while creating databricks custom app
I am trying to create a simple databricks custom app but I am getting Error: Could not import 'app'. error.app.yaml fileenv: - name: FLASK_APP value: '/Workspace/Users/sam@xxx.com/databricks_apps/hello-world_2025_11_13-16_19/Gaap_commentry/app'comm...
- 79 Views
- 2 replies
- 0 kudos
- 0 kudos
Thanks, I have modified the yaml file but still getting Error: Could not import 'app' errorenv: - name: FLASK_APP value: '/Workspace/Users/xxx@zzz.com/databricks_apps/hello-world_2025_11_13-16_19/Gaap_commentry'command: [ "flask", "--app", "...
- 0 kudos
- 51 Views
- 1 replies
- 2 kudos
Usage Dashboard from Cody Austin Davis displays only DBUs or overall cost including the VM uptime?
Hello,I have been looking at the new Dashboard created by @CodyA ( great job! ) and I was wondering if the cost displaying only provides visibility to only the databricks mark-up on each job ( i.e. $ DBUs ) or to the overall cost including cloud prov...
- 51 Views
- 1 replies
- 2 kudos
- 2 kudos
As the dashboard uses System Tables (system.billing.usage) to show spend across Jobs, SQL, and Notebooks, I dont beleive it includes cloud provider VM costs.
- 2 kudos
- 61 Views
- 4 replies
- 1 kudos
Want to see logs for lineage view run events
Hi All,I need your help, as I am running jobs it is getting successful, when I click on job and there we can find lineage > View run events option when click on it. I see below steps.Job Started: The job is triggered.Waiting for Cluster: The job wait...
- 61 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi Team/Member,As I am running jobs it is getting successful, when I click on job and there we can find lineage > View run events option when click on it. We find below steps and also added screenshot of it. I want screenshot stages logs, where i wil...
- 1 kudos
- 110 Views
- 4 replies
- 4 kudos
Resolved! Autoloader Managed File events
Hi all,We are in the process of migrating from directory listing to managed file events in Azure Databricks. Our data is stored in an Azure Data Lake container with the following folder structure:To enable file events in Unity Catalog (UC), I created...
- 110 Views
- 4 replies
- 4 kudos
- 4 kudos
Recommended approach to continue your existing pattern:Keep the External Location enabled for file events at the high-level path (/Landing).Run a separate Structured Streaming job for each table, specifying the full sub-path in the .load() function (...
- 4 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 492 | |
| 349 | |
| 287 |
