Hi Databricks Team,I'm trying to understand how Auto Loader Managed File Events work with Unity Catalog Managed Volumes, and I'm looking for some clarification.My understandingI create an External Location: CREATE EXTERNAL LOCATION app_ext_locURL 's3...
Hi team,I completed all 4 required courses for the recent Learning Festival campaign (June 15 – July 6, 2026):- DevOps Essentials for Data Engineering- Build Data Pipelines with Lakeflow Spark Declarative Pipelines- Deploy Workloads with Lakeflow Job...
Environment:Bundle engine: directResource type: Volume (Unity Catalog)Problem:I'm trying to use the lifecycle field with prevent_destroy: true on a Unity Catalog volume resource in my Declarative Automation Bundle (DAB). the validation fails with:Err...
Hi, The configuration is valid and the CLI version supports the lifecycle field. However, the error is most likely coming from outdated Databricks Python packages in your virtual environment at /home/animesh/datazone/.venv/. CLI uses the packages fro...
Hi there,Im new to Databricks and trying to build a SCD2 type table using AUTO CDC approach. while it quite simple to create a scd2 table Im unable to do a backfill.Full context.I have raw data(order, customer info) from 2019 and creating a dimension...
Hi @JD18, Welcome to Databricks and Thanks for raising this. SCD Type 2 backfilling with streaming tables is a common need, and the good news is that the AUTO CDC framework (formerly APPLY CHANGES INTO) has built-in capabilities to handle this -- you...
Hello,I am authenticated as GCP service account. I have to create query, run job as service principle and get result.I'm both cases, I set permissions CAN_MANAGE for group "users" for both query and job. When I create the query with RunAsMode.OWNER, ...
Hello @balajij8,I tried to use SQL File Task and was able to achieve the result. Only issue was, that I was not able to get output using WorkspaceClient.jobs.get_run_output(). It worked will direct call of /api/2.2/jobs/runs/get-output endpoint.Thank...
We're in the process of converting over our Databricks notebooks from .py file to .ipynb. We have disabled storing notebook output in source control at the workspace level.However, what we're discovering is that every cell in our notebooks has 3 time...
Hi @emorgoch, Thanks for raising this. This appears to be a regression rather than expected behaviour. Internally, the issue has been identified around .ipynb handling in Git folders, and the intended fix is to stop serialising these execution timest...
Hi Folks,My understanding isSELECT *FROM table_changes('mytable', x)should return all changes from version x through the latest available version, regardless of when I run it. However what happen to me is:When immediately after a streaming table upda...
Thank you, Ashwin, for looking into my question.My table is appended whenever a new file arrives. Each file contains the same number of records (8,676), but with a different extraction date.Regarding the results of your query above, two additional fi...
I have 2 service principals for deployment and runtime like databricks suggested, since declarative asset bundles dont include delta tables creation what should i just let my runtime to create table on the first run of a specific job?I want the commu...
You can configure the runtime SP to create tables dynamically on the first run if you prioritize operational simplicity and self-contained pipelines. The data processing scripts inject CREATE TABLE IF NOT EXISTS statements just before executing Data ...
Hi Everyone,Today I gave databricks exam for and I got 64 questions and my result was exactly 70.00%(As per databricks the pass percentage is 70 or above). but still the status was showing Failed and I couldn't get certified.Can you anyone help me on...
That sounds really bad, especially since your score is exactly 70 percent. I would wait for the Databricks team to determine whether the issue is with scoring or reporting because you have already created a support ticket. Hopefully they can clarify ...
Hello @Jim ,I completed all four modules of Learning Pathway 1 (Associate Data Engineering) for the Advanced Learning Festival on July 6, 2026, the published end date of the event. My Academy account email is [datasci.patel.rupesh@gmail.com]. Screens...
Hi @rjcrystal786, Thanks for checking. Based on the published guidance for the Advanced Learning Festival, if all four modules in Learning Pathway 1 were completed within the event window, your 50% certification voucher will be sent to the email addr...
I've recently started using the Databricks Standard SharePoint connector within my workspace and have run into some significant performance issues.My notebook does a straightforward read using the following:lakeflow_connection_name = 'sharepoint_dev'...
Hi all, thanks for the help.Unfortunately pathGlobFilter didn't improve performance for me.I ended up working around it with some pre-processing to figure out which specific folders I actually needed, then looping through those directories with the c...
I am a beginner for Data Bricks. Want to learn from the scratch.
Hi @Shaily_DBA, If you're new to Databricks and want to start from scratch, I'd recommend beginning with Databricks Free Edition. It’s free to use, designed for learners and aspiring data/AI professionals, and gives you a hands-on environment to star...
We offer the best web hosting solutions that are blazing fast, and ultra reliable & our sales & support team is here to help you find the right solutions
Thanks for sharing this information. A reliable hosting provider is important for any service business website. We recently optimized the Kitchen Chimney Service Guwahati website for Mannubhai and found that fast hosting significantly improves page s...
AI only matters when it improves workflows, speeds up decisions, and creates measurable business value.So, what would real AI impact look like inside your company?
hello everyone, I'm trying use spotify's api to analyse my music data, but i'm receiving a error during authentication, specifically when I try get the token, above my code.Is it a databricks bug?pip install spotipyfrom spotipy.oauth2 import SpotifyO...
I really enjoyed reading it and found the insights both practical and engaging. It's always great to come across content that provides real value to readers. I also enjoy exploring articles on digital trends, lifestyle, and informative topics at Con...
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 576 | |
| 349 | |
| 287 |
