Join us for the return of the Virtual Learning Festival! Mark your calendars from 9 April - 30 April 2025! Upskill across data engineering, data analysis, machine learning, and generative AI. Join the thousands who have elevated their career with...
- 13504 Views
- 46 replies
- 15 kudos
a month ago
Date & Time:8 April (Tuesday), 11 AM – 3 PM SGT Registration Page:Register Here: https://events.databricks.com/FY260408-SD-AIBIforSelf-ServiceAnalytics What You'll Learn:In this training session, you will learn how to self-serve business insights ...
- 1639 Views
- 1 replies
- 2 kudos
a week ago
Learn the foundational principles of the lakehouse architecture with three short self-paced videos The architecture of your data platform directly affects the reliability, performance and scalability of your data and AI initiatives. The lakehouse arc...
- 2520 Views
- 0 replies
- 2 kudos
4 weeks ago
Be part of a global movement! Connect with 22,000 data enthusiasts at 700+ sessions, keynotes, and hands-on training at this year’s Data + AI Summit. Whether you’re into data intelligence, governance, AI, or data warehousing, this is your chance to l...
- 3112 Views
- 0 replies
- 3 kudos
02-19-2025
Meet Takuya Omi (尾美拓哉), a passionate Data Engineer at NS Solutions and a valuable member of the Databricks Community. Takuya's career journey began in the marketing division of a financial institution, where his early work with SQL and machine learni...
- 317 Views
- 4 replies
- 6 kudos
Monday
Community Activity
- 281 Views
- 4 replies
- 1 kudos
Recovering Lost Dashboard Data
Hello,I attempted to update the SQL warehouse ID for all our dashboards using the Databricks SDK lakeview.update method. However, I mistakenly passed a dashboard object without the serialized_dashboard value, which resulted in all our dashboards beco...
- 281 Views
- 4 replies
- 1 kudos
- 261 Views
- 6 replies
- 2 kudos
CANNOT_UPDATE_TABLE_SCHEMA
I'm encountering a puzzling schema merge issue with my Delta Live Table. My setup involves several master tables on Databricks, and due to a schema change in the source database, one of my Delta Live Tables has a column (e.g., "reference_score") that...
- 261 Views
- 6 replies
- 2 kudos
Latest Reply
- 2 kudos
Hey Hung, before dropping the table, there are a few other things you could try. One option is to point your DLT logic to a new table with the updated FLOAT column, just to confirm everything works with the new schema—this helps isolate whether the i...
- 2 kudos
- 25 Views
- 2 replies
- 0 kudos
Spark DataFrame Checkpoint
Good morning,I am having a difficulty when trying to checkpoint a PySpark DataFrame.The DataFrame is not involved in a DLT pipeline so I am using the df.checkpoint(eager=True) command, to truncate the logical plan of df and materialize it as files wi...
- 25 Views
- 2 replies
- 0 kudos
- 55 Views
- 3 replies
- 0 kudos
databricks_error_message:REQUEST_LIMIT_EXCEEDED:
A databricks job failed unexpectedly with below error. There ware only 5 jobs running at that time and no major operations. what could be the root cause and how can we avoid this in futureCluster '0331-xxxxxx-zs8i8pcn' was terminated. Reason: INIT_SC...
- 55 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
are you generating any databricks tokens in this process? if yes there is a limit of 600.
- 0 kudos
- 7960 Views
- 24 replies
- 2 kudos
Data Engineer Associate Certification in 2025 Voucher
Hi Community,I’m planning to take the Data Engineer Associate Certification in 2025 to advance my skills and career. Are there any ongoing promotions or opportunities to get a 50% discount voucher for the exam? It would be incredibly helpful if you c...
- 7960 Views
- 24 replies
- 2 kudos
- 109 Views
- 8 replies
- 4 kudos
Performance issue with Spark SQL when working with data from Unity Catalog
We're encountering a performance issue with Spark SQL when working with data from Unity Catalog. Specifically, when I use Spark to read data from a Unity Catalog partition created by DLT and then create a view, the executor retrieval is very slow. ...
- 109 Views
- 8 replies
- 4 kudos
Latest Reply
- 4 kudos
If there is only a small difference between the number of files, that is probably not it (unless there is serious data skew).DLT probably adds this config for data quality reasons (which might be defined on the dlt). But as I do not use dlt, I could...
- 4 kudos
- 48 Views
- 1 replies
- 0 kudos
Exam got suspended for violating absolutely nothing
Hi @Cert-TeamOPS , Exam got suspended for violating absolutely nothing. This is pathetic harassment. Check the video.Reschedule or refund.
- 48 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello, @jarbed001! It looks like you've already asked this question in an existing thread, where it has been answered. Please check that thread for the response and continue the discussion there to keep the conversation focused and organized.
- 0 kudos
- 27 Views
- 0 replies
- 0 kudos
Found a potential bug in Job Details/Schedule and Trigger section
One of our jobs is scheduled to run at 4:30 AM based on GMT+1 timezone, which is visible if we click on the Edit trigger (Picture1), but under job details it is show as if it was schedule to run at 4:30 AM UTC time (Picture 2).Based on previous runs ...
- 27 Views
- 0 replies
- 0 kudos
- 193 Views
- 3 replies
- 0 kudos
Disabled billing system schema accidentally and now I cannot enable it again
We accidentally disabled billing system table schema and now getting this error on re-enabling it:Error: billing system schema can only be enabled by Databricks.How can we resolve the same. We have not purchased any of the support contracts and hence...
- 193 Views
- 3 replies
- 0 kudos
- 20 Views
- 0 replies
- 0 kudos
Implementing Governance on DLT pipelines using compute policy
I am implementing governance over compute creation in the workspaces by implementing custom compute policies for all-purpose, job and dlt pipelines. I was successfully able to create compute policies for all-purpose and jobs where I could restrict th...
- 20 Views
- 0 replies
- 0 kudos
- 498 Views
- 2 replies
- 0 kudos
Convert Account to Self-managed
I am in the process of setting up a new Databricks account for AWS commercial. I mistakenly setup the account with the email: databricks-external-nonprod-account-owner@slingshotaerospace.com to not be self-managed and I would like for this new accoun...
- 498 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Or better yet if we could delete it so I can re-create the account.
- 0 kudos
- 40 Views
- 1 replies
- 0 kudos
Databricks certification Voucher Needed
Is there any program from databricks where we can get the 100% voucher for the certification by completing any particular course/training from databricks, can you please help with this information ?could you please provide me voucher for "Databricks ...
- 40 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello @Pjain31 ,You are in luck! We are currently running a Learning Festival with a certification voucher. Please follow below link for details:https://community.databricks.com/t5/events/virtual-learning-festival-9-april-30-april/ev-p/111620Learning...
- 0 kudos
- 177 Views
- 1 replies
- 0 kudos
Resolved! dbutls.fs.cp() fails in Runtime 16.3 Beta, when using abfss://
Hello,I am not sure if this is the right place to post this, however, I reporting what seems to me, a breaking issue with 16.3 Beta Runtime, when performing dbutils.fs.cp() operations between abfss://This is not a permissions issue — let's get that o...
- 177 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
To close off my loop: The ClassCastException has been resolved in 16.3
- 0 kudos
- 195 Views
- 1 replies
- 0 kudos
Change Data Feed And Column Masks
Hi there,Wondering if anyone can help me. I have had a job set up to stream from one change data feed enabled delta table to another delta table and has been executing successfully. I then added column masks to the source table from which I am stream...
- 195 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello @mh177! It looks like this post duplicates the one you recently posted. A response has already been provided to that post. I recommend continuing the discussion in that thread to keep the conversation focused and organised.
- 0 kudos
- 35938 Views
- 140 replies
- 9 kudos
Suspended Exam? Don't worry! Our support team can help!
Taking exams is so stressful already, and if your exam gets suspended it is extra stressful. Don't worry! Our support team is awesome and will help you get back on track. The most important step is to file a ticket in our Help Center. Be sure to inc...
- 35938 Views
- 140 replies
- 9 kudos
Latest Reply
- 9 kudos
Hi @Cert-Team @Cert-TeamOPS @DB_Paul ,My exam got suspended for no reason, please help resolve this by rescheduling.The questions come on the left of the screen, so I was looking slightly left to read and answer them. Suddenly I was asked not to look...
- 9 kudos
Announcements
| User | Count |
|---|---|
| 106 | |
| 43 | |
| 22 | |
| 17 | |
| 14 |
Latest from our Blog
How to Assess and Adopt New Technologies Using Data-Driven Proof of Concept
In this article, I share a #data-driven method for assessing and adopting new technologies and steps for creating or evaluating a strong proposal. This includes using statistical measures to evaluate ...
[PARTNER BLOG] Introducing Databricks Native Master Data Management (MDM) - Entity Resolution
LakeFusion’s MDM solution delivers a single source of truth by leveraging advanced entity resolution and deduplication algorithms. By unifying fragmented data across systems, we ensure accurate, consi...
💡 ML Training Tip Of The Week #5: Enable fine-grained data frequency in AutoML forecast
Customers with high-frequency time series data often need forecasts at a finer granularity based on their data sources. For example, an energy supply company may receive energy demand data every 15 mi...
💡 ML Training Tip Of The Week #4: Speed up your ML workload with one toggle
Machine learning workload can be time-consuming due to training data processing such as ETL and feature engineering and iterative model training steps. Recent ML Runtime releases have made both faster...
