- 7481 Views
- 11 replies
- 0 kudos
Databricks apps - Volumes and Workspace - FileNotFound issues
I have a Databricks App I need to integrate with volumes using local python os functions. I've setup a simple test: def __init__(self, config: ObjectStoreConfig): self.config = config # Ensure our required paths are created ...
- 7481 Views
- 11 replies
- 0 kudos
- 0 kudos
These are the permissions needed for the app service principal:Catalog: Use CatalogSchema: Use Schema Volume: Read / WriteTo actually be able to read/write in the volume I had to use the Python databricks-sdk. Performing "normal" file system operatio...
- 0 kudos
- 2202 Views
- 3 replies
- 3 kudos
Databricks community group in Kerala
Calling All Data Enthusiasts in Kerala! Hey everyone,I'm excited about the idea of launching a Databricks Community Group here in Kerala! This group would be a hub for learning, sharing knowledge, and networking among data enthusiasts, analysts, a...
- 2202 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @AswinGovindan77 , Do you have an estimate of how many enthusiasts will be joining you? If you provide a rough number, we can create a group for you on Community!
- 3 kudos
- 620 Views
- 1 replies
- 0 kudos
Resolved! DQX Compatibility
Hi all,Does anyone know if DQX (Databricks Labs) is compatible with Microsoft Fabric Spark notebooks? It would be useful to know this, so that we can use it on Fabric-based projects as well.Thanks!
- 620 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @d_a_n, No, DQX isn’t supported on Microsoft Fabric Spark notebooks. DQX is a Databricks Labs framework that’s designed and documented to run on Databricks workspaces and Databricks clusters only (see the official installation prerequisites). In a...
- 0 kudos
- 4605 Views
- 11 replies
- 4 kudos
Resolved! Azure Databricks session expiry Issue
I frequently encounter a session-expired issue while using Databricks within 2 minutes, and this has been happening for two days. Are there any fixes or suggestions?
- 4605 Views
- 11 replies
- 4 kudos
- 4 kudos
Hi @Prathy @prasadv @joonkim4078 @Krissy_ , Thank you for your patience while we investigated this issue. We flagged this to our teams internally, and the team has deployed a fix, and based on our checks, the behaviour should now be back to normal. T...
- 4 kudos
- 928 Views
- 1 replies
- 2 kudos
Resolved! Learn scala using Databricks free edition
Hi, I want to run and learn spark scala using databricks free edition. It looks there was access in community edition but now in databricks free edition serverless does not scala . Please confirm if understanding is correct and possible alternative i...
- 928 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @kpavan2004, You are right. The free edition doesn't support R or Scala. You can see a list of limitations here. If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix...
- 2 kudos
- 1200 Views
- 2 replies
- 1 kudos
Serverless Compute Idle Timeout Time
I want to know what is the idle timeout for Databricks Serverless Compute. Can anyone Databricks document where it specifies the idle timeout time details.
- 1200 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @AJ270990 , I did some digging and I have some helpful tips to guide you. This is one of those areas where the answer is “it depends,” based on which flavor of serverless compute you’re using. Let’s break it down so it’s easy to reason about. Firs...
- 1 kudos
- 2008 Views
- 3 replies
- 1 kudos
Resolved! Guidance on Databricks Specializations and Migration Accelerators for Partner Tier Progression
Hi everyone,We are currently working toward progressing our tier in the Databricks Partner Network and had a quick question regarding the tier requirements.Could someone clarify:• What is the process for partners to achieve Databricks Specializations...
- 2008 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @vamsi_simbus, Thanks for the questions and for growing your Databricks practice as you move beyond Bronze in the Databricks Partner Network. At a high level, each Brickbuilder Specialisation (industry or product) has a clear checklist that typica...
- 1 kudos
- 4604 Views
- 4 replies
- 12 kudos
Resolved! Blueprint: Entity Resolution(Dedup & Golden Records)on Databricks — blocking, fuzzy scoring, MLflow
Hi Databricks Community I’m working on designing a practical Entity Resolution / Deduplication framework on Databricks, and I wanted to share a high‑level blueprint and learn from others who’ve tackled similar problems at scale.The use case is fairly...
- 4604 Views
- 4 replies
- 12 kudos
- 12 kudos
Hi @Mridu, Rather than building everything greenfield, you can lean heavily on Databricks’ existing Entity Resolution Solution Accelerators (Customer ER, Product Matching, Public Sector ER) as a starting point and then adapt their patterns to your bl...
- 12 kudos
- 3298 Views
- 2 replies
- 0 kudos
Databricks workspace in failed state
Hi Community,I had my Databricks workspace up and running and it was managed through terraform, and encryption was enabled through cmk, there were some updation in the code, so I put terraform plan, one of the key changes(replace) it showed me was"az...
- 3298 Views
- 2 replies
- 0 kudos
- 1392 Views
- 5 replies
- 12 kudos
Resolved! LLM based Replies to community posts
Hello Everyone,I’ve been in the community for a few weeks and really enjoy reading the problems and production-focused solutions shared here.Recently, I’ve noticed some replies(solutions) that look quite verbose and seem like they might be generated ...
- 1392 Views
- 5 replies
- 12 kudos
- 12 kudos
Hey guys, I appreciate you bringing this up. We do have an AI usage policy in our Code of Conduct that I recently asked Legal to update:Use of AI Tools: AI tools may be used to assist with formatting, clarity, and organization, or to draft ideas that...
- 12 kudos
- 689 Views
- 4 replies
- 0 kudos
Attempting to load a JSON file fails due to schema issue (Free Edition)
Hello,I created a Volume named 'test_volume' under catalog:workspace and schema:default.Then I uploaded a file named user_0.json into test_volume (fake data, of course):Now I want to load that file into a data frame.With Python in a notebook:With SQL...
- 689 Views
- 4 replies
- 0 kudos
- 0 kudos
The AnalysisException you're seeing in the Databricks Community Edition is almost always caused by a mismatch between the JSON file format and Spark’s default reader.By default, Spark expects JSON Lines (one JSON object per line). If your file is a s...
- 0 kudos
- 1951 Views
- 5 replies
- 3 kudos
Resolved! “Provisioned throughput is not enabled for this workspace”
Hello,when I try setting up a model with provisioned throughput, the deployment fails with the message “Provisioned throughput is not enabled for this workspace.”. It doesn’t work for databricks-hosted models nor for 3rd party models from the marketp...
- 1951 Views
- 5 replies
- 3 kudos
- 3 kudos
Hello Steve,thank you for the broad reply. Just FYI I had to reach to to Azure support and DB support to fix it.FYI 2: Serverless compute was enabled, `Enforce data processing within workspace Geography for Designated Services was disabled`, pay-per-...
- 3 kudos
- 1132 Views
- 2 replies
- 0 kudos
Resolved! How to auto‑terminate DLT-managed clusters after pipeline execution?
We have Data bricks Jobs that run a combination of DLT pipelines and notebook tasks.For the notebook tasks, we use a job cluster, and we are able to auto‑terminate it after execution using auto termination: 10However, the DLT-managed clusters behave ...
- 1132 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @anusha98, The behavior you are seeing, where the cluster stays running for a long time after pipeline completion, is almost certainly because your pipeline is running in Development mode. In Development mode, the default cluster shutdown delay is...
- 0 kudos
- 599 Views
- 2 replies
- 1 kudos
Manager hierarchy
for employee manager hierarchy some employees are not having correct manager mapping, what can be the reason?
- 599 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @spjti, Incorrect manager mappings in an employee hierarchy typically come from a few common root causes. Since the question is fairly open-ended, here is a checklist of the most likely issues and how to diagnose them in Databricks. COMMON REASONS...
- 1 kudos
- 1216 Views
- 6 replies
- 0 kudos
Same Delta Table, Different Behavior: Dev vs Prod Workspace in Databricks
I recently ran into an interesting Databricks behavior while implementing a row-count comparison using Delta Time Travel (VERSION AS OF).Platform: AzureScenario:Same Unity CatalogSame fully qualified tableSame table ID, location, and Delta formatYet ...
- 1216 Views
- 6 replies
- 0 kudos
- 0 kudos
Hi @Smriti2, The error "TIME TRAVEL is not allowed. Operation not supported on Streaming Tables" when the table is not intentionally a streaming table typically points to the table having pipeline-related metadata attached to it in the prod workspace...
- 0 kudos
