- 162 Views
- 3 replies
- 0 kudos
Permissions on job with SQL task
Hello,I am authenticated as GCP service account. I have to create query, run job as service principle and get result.I'm both cases, I set permissions CAN_MANAGE for group "users" for both query and job. When I create the query with RunAsMode.OWNER, ...
- 162 Views
- 3 replies
- 0 kudos
- 0 kudos
Hello @balajij8,thank you for your answer.Today I did testing with setup:Job Creator: GCP SAJob RunAs: SPQuery Owner: Personal accountPermissions: CAN_MANAGE for all users for both Query and JobBoth personal account and GCP SA were able to get the jo...
- 0 kudos
- 464 Views
- 3 replies
- 2 kudos
Resolved! Managing IPYNB cell timestamps in source control
We're in the process of converting over our Databricks notebooks from .py file to .ipynb. We have disabled storing notebook output in source control at the workspace level.However, what we're discovering is that every cell in our notebooks has 3 time...
- 464 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @emorgoch, Thanks for raising this. This appears to be a regression rather than expected behaviour. Internally, the issue has been identified around .ipynb handling in Git folders, and the intended fix is to stop serialising these execution timest...
- 2 kudos
- 187 Views
- 2 replies
- 0 kudos
select table_changes() return unexpected results
Hi Folks,My understanding isSELECT *FROM table_changes('mytable', x)should return all changes from version x through the latest available version, regardless of when I run it. However what happen to me is:When immediately after a streaming table upda...
- 187 Views
- 2 replies
- 0 kudos
- 0 kudos
Thank you, Ashwin, for looking into my question.My table is appended whenever a new file arrives. Each file contains the same number of records (8,676), but with a different extraction date.Regarding the results of your query above, two additional fi...
- 0 kudos
- 80 Views
- 1 replies
- 0 kudos
Declarative Asset Bundle Service Principals best practices
I have 2 service principals for deployment and runtime like databricks suggested, since declarative asset bundles dont include delta tables creation what should i just let my runtime to create table on the first run of a specific job?I want the commu...
- 80 Views
- 1 replies
- 0 kudos
- 0 kudos
You can configure the runtime SP to create tables dynamically on the first run if you prioritize operational simplicity and self-contained pipelines. The data processing scripts inject CREATE TABLE IF NOT EXISTS statements just before executing Data ...
- 0 kudos
- 44288 Views
- 13 replies
- 14 kudos
Resolved! I Got 70.00% on Databricks Certified Data Engineer Professional Exam but Failed....
Hi Everyone,Today I gave databricks exam for and I got 64 questions and my result was exactly 70.00%(As per databricks the pass percentage is 70 or above). but still the status was showing Failed and I couldn't get certified.Can you anyone help me on...
- 44288 Views
- 13 replies
- 14 kudos
- 14 kudos
That sounds really bad, especially since your score is exactly 70 percent. I would wait for the Databricks team to determine whether the issue is with scoring or reporting because you have already created a support ticket. Hopefully they can clarify ...
- 14 kudos
- 148 Views
- 1 replies
- 0 kudos
Bundle Validation Error: Volume lifecycle Field Rejected Despite Being in Schema
Environment:Bundle engine: directResource type: Volume (Unity Catalog)Problem:I'm trying to use the lifecycle field with prevent_destroy: true on a Unity Catalog volume resource in my Declarative Automation Bundle (DAB). the validation fails with:Err...
- 148 Views
- 1 replies
- 0 kudos
- 0 kudos
What you are seeing is expected behavior when the local execution environment is out of sync with the latest target schema definitions. prevent_destroy: true is a fully supported property for Unity Catalog Volume resources within Databricks Asset Bun...
- 0 kudos
- 155 Views
- 1 replies
- 0 kudos
Advanced Learning Festival (15 June - 06 July 2026) - Voucher Eligibility Confirmation
Hello @Jim ,I completed all four modules of Learning Pathway 1 (Associate Data Engineering) for the Advanced Learning Festival on July 6, 2026, the published end date of the event. My Academy account email is [datasci.patel.rupesh@gmail.com]. Screens...
- 155 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @rjcrystal786, Thanks for checking. Based on the published guidance for the Advanced Learning Festival, if all four modules in Learning Pathway 1 were completed within the event window, your 50% certification voucher will be sent to the email addr...
- 0 kudos
- 369 Views
- 4 replies
- 4 kudos
Databricks Standard SharePoint Connector Performance Issues
I've recently started using the Databricks Standard SharePoint connector within my workspace and have run into some significant performance issues.My notebook does a straightforward read using the following:lakeflow_connection_name = 'sharepoint_dev'...
- 369 Views
- 4 replies
- 4 kudos
- 4 kudos
Hi all, thanks for the help.Unfortunately pathGlobFilter didn't improve performance for me.I ended up working around it with some pre-processing to figure out which specific folders I actually needed, then looping through those directories with the c...
- 4 kudos
- 137 Views
- 2 replies
- 0 kudos
Getting start with Data Brick machine creation.
I am a beginner for Data Bricks. Want to learn from the scratch.
- 137 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Shaily_DBA, If you're new to Databricks and want to start from scratch, I'd recommend beginning with Databricks Free Edition. It’s free to use, designed for learners and aspiring data/AI professionals, and gives you a hands-on environment to star...
- 0 kudos
- 1182 Views
- 1 replies
- 0 kudos
hosting-services-dedicated-shared-colocation-dubai-uae-middle-east-saudi-arabia-oman-qatar-bahrain-kuwait (1)
We offer the best web hosting solutions that are blazing fast, and ultra reliable & our sales & support team is here to help you find the right solutions
- 1182 Views
- 1 replies
- 0 kudos
- 0 kudos
Thanks for sharing this information. A reliable hosting provider is important for any service business website. We recently optimized the Kitchen Chimney Service Guwahati website for Mannubhai and found that fast hosting significantly improves page s...
- 0 kudos
- 77 Views
- 0 replies
- 0 kudos
Is your AI strategy driving real impact or just more pilots?
AI only matters when it improves workflows, speeds up decisions, and creates measurable business value.So, what would real AI impact look like inside your company?
- 77 Views
- 0 replies
- 0 kudos
- 19207 Views
- 27 replies
- 4 kudos
spotify API get token - raw_input was called, but this frontend does not support input requests.
hello everyone, I'm trying use spotify's api to analyse my music data, but i'm receiving a error during authentication, specifically when I try get the token, above my code.Is it a databricks bug?pip install spotipyfrom spotipy.oauth2 import SpotifyO...
- 19207 Views
- 27 replies
- 4 kudos
- 4 kudos
I really enjoyed reading it and found the insights both practical and engaging. It's always great to come across content that provides real value to readers. I also enjoy exploring articles on digital trends, lifestyle, and informative topics at Con...
- 4 kudos
- 80 Views
- 0 replies
- 1 kudos
from_utc_timestamp silently double-shifts time when session timezone isn't UTC — docs should call th
Summary:from_utc_timestamp(current_timestamp(), '<tz>') produces an incorrect (future-shifted) timestamp whenever the Spark session timezone is already set to something other than UTC. This is a very common pattern for teams stamping dp_load_ts/creat...
- 80 Views
- 0 replies
- 1 kudos
- 150 Views
- 1 replies
- 1 kudos
Lakebase CDF — destination Delta table not created after successful UI setup (Free Edition)
Summary:We configured Lakebase CDF to stream changes from a native Postgres table (public.query_logs) into Unity Catalog, but the destination Delta table never appears despite completing all documented prerequisites.Steps taken1. Created query_logs ...
- 150 Views
- 1 replies
- 1 kudos
- 1 kudos
Most likely this is a Free Edition storage limitation rather than a bug, and your Postgres-side setup sounds correct. Lakebase CDF writes its destination as a Unity Catalog managed Delta table, and the Lakebase CDF docs list catalogs backed by defaul...
- 1 kudos
- 13428 Views
- 7 replies
- 2 kudos
Relevance of off heap memory and usage
I was referring to the doc - https://kb.databricks.com/clusters/spark-executor-memory.In general total off heap memory is = spark.executor.memoryOverhead + spark.offHeap.size. The off-heap mode is controlled by the properties spark.memory.offHeap....
- 13428 Views
- 7 replies
- 2 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 576 | |
| 349 | |
| 287 |
