- 657 Views
- 1 replies
- 0 kudos
Requesting free/discounted voucher for Data Engineer Associate certification exam
Hi everyone,I'm currently preparing for the Databricks Certified Data Engineer Associate exam and have completed the self-paced training on Databricks Academy. I'm a [student / early-career professional / etc. — fill in your situation] and the exam f...
- 657 Views
- 1 replies
- 0 kudos
- 0 kudos
@Rahulkumar65 Vouchers are distributed to the eligible participants after the Learning Festival. Lookout for the next one in October. If your organisation is a partner of Databricks, then check with your admin.
- 0 kudos
- 155 Views
- 1 replies
- 1 kudos
Lakehouse Monitoring Solution
I wonder if there is any GitHub that specifically focuses on Databricks' Lakehouse monitoring solution with definitions and codes that can do this.I want to onboard it for a customer of mine and don't want to write from scratch if one is already pres...
- 155 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi Surya, You can begin with Data Quality Monitoring solution - already present. It has anomaly detection & Data Profiling (formerly known as Lakehouse Monitoring) solutions helping in Statistical Profiling, data quality, snapshot quality monitors, d...
- 1 kudos
- 230 Views
- 1 replies
- 1 kudos
Synced table pipeline fails with permission denied for database
Environment: Azure Databricks, Lakebase Autoscaling project. Database created through the Lakebase UI by the project owner (audit log shows createProjectBranchDatabaseUI with a browser user agent). Synced tables created via the databricks_postgres_sy...
- 230 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @KhaturiabPreet !I like you analysis it made me do some reading this morning to freshen up my memory.I checked the the doc and I can say that databricks_writer_<dbid> being unable to CONNECT does not look like an expected state because it is t...
- 1 kudos
- 213 Views
- 2 replies
- 3 kudos
Databricks Support #00984257 - Password Reset Email is not working for me
Hello Everyone,I had raised a support request with the DBX Support team on Wednesday, 5 August 2026, but I have not received any response yet.Could anyone please guide to resolve my login issue? I forgot my password, and when I click the "Forgot Pass...
- 213 Views
- 2 replies
- 3 kudos
- 307 Views
- 1 replies
- 3 kudos
Resolved! Is it safe to expose JWT in Databricks Job?
There is an usecase where I want to verify/extract the job caller identity in the job runtime of databricks. The job run_as and creator parameter seems to be fixed at the job configuration and doesn't reflect/update who actually triggered a job run.A...
- 307 Views
- 1 replies
- 3 kudos
- 3 kudos
Hello !What I can say is that your conclusion is correct since DBKS does not currently expose the identity of the principal that called run-now to the running job through the Jobs API or notebook context and the audit event is the authoritative sourc...
- 3 kudos
- 529 Views
- 2 replies
- 0 kudos
Resolved! Support Multiple Tasks DAG Inside a `for_each_task` Iteration
Databricks Jobs `for_each_task` accepts exactly one nested task. For workloads that run the *samemulti-step pipeline over many datasets* — a very common metadata-driven pattern — this forces achoice between two architectures, each of which gives up s...
- 529 Views
- 2 replies
- 0 kudos
- 0 kudos
Hey droid,First — your analysis is correct and still current. I re-checked the docs: a ForEach task takes exactly one nested task (you can't even nest another ForEach), and the dependency scope sits at the ForEach level, so you can't fan out a hetero...
- 0 kudos
- 4484 Views
- 4 replies
- 3 kudos
Asset Bundles: Shared libraries and notebooks in monorepo multi-bundle setup
I am part of a small team of Data Engineers which started using Databricks Asset Bundles one year ago. Our code base consists of typical ETL-workloads written primarily in Jupyter notebooks (.ipynb), and jobs (.yaml) with our codebase spanning across...
- 4484 Views
- 4 replies
- 3 kudos
- 3 kudos
Your proposed structure is sensible — it is the standard pattern for large-team monorepo setups with Databricks Asset Bundles. Here is a more complete answer to both questions you raised.1. Shared Python libraries: build a wheel, wire it as a bundle ...
- 3 kudos
- 230 Views
- 2 replies
- 1 kudos
Looking for recommendations
Does anyone know a good tool thats can generate syenthetic time series-data from existing datasets?I am exploring options for data augmentation,testing and experimenting with different models.Open-source or commercial-both workWould love to hear, wha...
- 230 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @ThiamLee , How are you doing today?As per my experience, I’d suggest giving Genie Code a try, especially if you’re already working in Databricks.I’ve been using Genie Code while building a few of my own Data + AI apps and POCs, and what I like mo...
- 1 kudos
- 204 Views
- 1 replies
- 0 kudos
available virtual machines in databricks
My team is investigating the migration of older and now non-adjustable azure skus to newer versions such as v5. I see in azure Usage+Quotas the Standard Dldsv5 and Dldsv6 families with available quota. However, when creating compute in Databricks the...
- 204 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @bill-kotronis ,The behavior you're seeing is expected — not all Azure VM families are exposed in Databricks, even if you have quota for them.Databricks maintains a curated, validated subset of Azure instance types. Having available quota in Azure...
- 0 kudos
- 450 Views
- 2 replies
- 2 kudos
Resolved! Critical Genie Agents Issue – Catalog Rename Causes Tables and Joins Loss
I have identified a serious issue in Genie Agents that requires attention from the Databricks Product Team.When a catalog is renamed, the tables that were previously configured and used within the Genie Agent disappear unexpectedly. After manually re...
- 450 Views
- 2 replies
- 2 kudos
- 256 Views
- 1 replies
- 2 kudos
Google Cloud Lakehouse catalog federation (Beta): queries fail with "Unrecognized...."
Has anyone gotten the new **Google Cloud Lakehouse catalog federation** Beta (GCP workspaces) to actually run queries?Setup (followed the docs end to end):- GCP workspace (us-central1, serverless-only), Unity Catalog- Previews page: Google Cloud Lake...
- 256 Views
- 1 replies
- 2 kudos
- 2 kudos
Hey matko,Short version: your diagnosis looks right, and this reads like a Beta rollout gap rather than anything you misconfigured.The stack trace (toProto failing in loadForeignSecurableMetadataAndUpdateCache on an unrecognized securable kind) plus ...
- 2 kudos
- 11794 Views
- 9 replies
- 2 kudos
Resolved! Trouble Enabling File Events For An External Location
Hello all,I am trying to enable file events on my Azure Workspace for the File Arrival Trigger trigger mode for Databricks Workflows. I'm following this documentation exactly (I think) but I'm not seeing the option to enable them. As you can see here...
- 11794 Views
- 9 replies
- 2 kudos
- 2 kudos
I am also facing some issue with the "file arrival" trigger on one of my environments.Its strange that it kept running fine until today where it broke. I always only had the "Storage Blob Data Contributor" for my Managed Identity ( Access Connector) ...
- 2 kudos
- 179 Views
- 1 replies
- 0 kudos
How to create an image from a cluster so that the compute environment can be replicated
I am working in a regulated industry. It is necessary to document and replicate the compute environment. I guess to do this for a classic cluster vs a serverless one is different. Anyone knows how to do it? Thank you for your assistance.
- 179 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @NW1000 ,For Classic Compute, the best approach is using Databricks Container Services (DCS). This lets you package your exact OS, system libraries, and Python environment into a custom Docker image. You just push that image to a private registry ...
- 0 kudos
- 297 Views
- 2 replies
- 1 kudos
Building a Visitor Data Pipeline for Digital Membership Card
Hi everyone, I'm working on a project where we collect visitor activity from multiple touchpoints such as ticketing, membership sign-ups, event participation, mobile app interactions, and digital membership card usage. The goal is to create a unified...
- 297 Views
- 2 replies
- 1 kudos
- 1 kudos
Concrete answer for the pieces you asked about, from a similar build (loyalty/membership 360 profile at bank scale - different domain, same shape of problem): Data model: model this as classic medallion, but the key decision is at silver/gold - use a...
- 1 kudos
- 2515 Views
- 8 replies
- 1 kudos
Resolved! How do I import a python module when deploying with DAB?
Below is how the folder structure of my project looks like: resources/ |- etl_event/ |- etl_event.job.yml src/ |- pipeline/ |- etl_event/ |- transformers/ |- transformer_1.py |- utils/ |- logger.py databricks.ym...
- 2515 Views
- 8 replies
- 1 kudos
- 1 kudos
You dont need to use wheel files . Use glob as the key instead of file - https://docs.databricks.com/aws/en/dev-tools/bundles/resources#pipelinelibrariesHere is the screenshot .
- 1 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 582 | |
| 349 | |
| 287 |
