Join us for the return of the Virtual Learning Festival! Mark your calendars from 9 April - 30 April 2025! Upskill across data engineering, data analysis, machine learning, and generative AI. Join the thousands who have elevated their career with...
- 5117 Views
- 22 replies
- 9 kudos
3 weeks ago
Date & Time:8 April (Tuesday), 11 AM – 3 PM SGT Registration Page:Register Here: https://events.databricks.com/FY260408-SD-AIBIforSelf-ServiceAnalytics What You'll Learn:In this training session, you will learn how to self-serve business insights ...
- 52 Views
- 0 replies
- 0 kudos
18 hours ago
We want to hear from you! Be among the first 20 people to share your experience with Databricks on G2 and receive a $50 gift card.Why Participate? Your Feedback Matters: It helps us innovate and improve.Quick and Easy: The review takes less than...
- 211 Views
- 0 replies
- 2 kudos
Wednesday
Learn the foundational principles of the lakehouse architecture with three short self-paced videos The architecture of your data platform directly affects the reliability, performance and scalability of your data and AI initiatives. The lakehouse arc...
- 1060 Views
- 0 replies
- 2 kudos
3 weeks ago
Be part of a global movement! Connect with 22,000 data enthusiasts at 700+ sessions, keynotes, and hands-on training at this year’s Data + AI Summit. Whether you’re into data intelligence, governance, AI, or data warehousing, this is your chance to l...
- 2160 Views
- 0 replies
- 3 kudos
02-19-2025
The Databricks Developer Relations team is excited to share that we are hitting the road and bringing a series of community meetups to you called DevConnect! These events will be hosted around the globe and tailored specifically for Data engineers. T...
- 4420 Views
- 2 replies
- 1 kudos
02-18-2025
Community Activity
- 75 Views
- 3 replies
- 2 kudos
Incrementalization issue in Materialized views
I am trying to implement the incremental updates to the Materialized views. The source is the Could you tell how to resolve the following issue? { "planning_information": { "technique_information": [ { "maintenance_type": "MAINTENANCE...
- 75 Views
- 3 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi Dharanip,From what you’ve described, your logic looks good, but the issue comes from how Materialized Views work in Databricks. They only support incremental updates when the query is very simple and fully predictable (or "deterministic"). Since y...
- 2 kudos
- 49 Views
- 1 replies
- 0 kudos
Inconsistent PYTHONPATH, Git folders vs DAB
Hello Databricks Community,I'm encountering an issue related to Python paths when working with notebooks in Databricks. I have a following structure in my project:my_notebooks - my_notebook.py /my_package - __init__.py - hello.py databricks.yml...
- 49 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi mydefaultlogin,How are you doing today?, As per my understanding, You're right—this happens because when you're running notebooks from your Git folder, Python knows exactly where your project root is and can easily find my_package. But when you de...
- 0 kudos
- 19 Views
- 1 replies
- 0 kudos
Change cluster owner API call failed
I am trying to change cluster's owner using API call. but get following error. I am positive that host, cluster_id and owner_username are all correct. The error message says No API found. Is this related with the compute I am using? or something else...
- 19 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi hdu,How are you doing today?, As per my understanding, It sounds like you’re really close! That “No API found” error usually means either the wrong API endpoint is being used, or the cluster type doesn’t support ownership changes—for example, shar...
- 0 kudos
- 8886 Views
- 3 replies
- 4 kudos
Materials to pass Databricks Data Engineering Associate Exam
Hi Guys, I have passed it already some time ago, but just recently have summarized all the materials which helped me to do it. Pay special attention to GitHub repository, which contains many great exercises prepared by Databricks teamhttps://youtu.be...
- 8886 Views
- 3 replies
- 4 kudos
Latest Reply
- 4 kudos
Best decision I made for my Databricks-Certified-Data-Engineer-Associate exam preparation! I used PASSEXAMHUB, and it saved me hours of unnecessary studying. Their practice tests gave me real exam experience, and I passed on my first try!
- 4 kudos
- 164 Views
- 4 replies
- 2 kudos
What's the recommended way of creating tables in Databricks with unity catalog (External/Managed)
I have databricks with unity catalog enables and created an external ADLS location. when I create the catalog/schema it uses the external location. when I try to create the table it uses the extrenal location but they are managed tables. What's the r...
- 164 Views
- 4 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi Shivap,How are you doing today?, As per my understanding, in Unity Catalog, if you want to create an external table, you just need to make sure the external location is registered and approved first. Even though you're specifying a path with LOCAT...
- 2 kudos
- 34548 Views
- 131 replies
- 9 kudos
Suspended Exam? Don't worry! Our support team can help!
Taking exams is so stressful already, and if your exam gets suspended it is extra stressful. Don't worry! Our support team is awesome and will help you get back on track. The most important step is to file a ticket in our Help Center. Be sure to inc...
- 34548 Views
- 131 replies
- 9 kudos
Latest Reply
- 9 kudos
@Cert-Team @Cert-TeamOPS Exam attempt suspended immediately upon launch. The links you post to view behavioral consideration and room requirements require the user to log in to Kryterion, and I've yet to see how to even create an account with them. I...
- 9 kudos
- 840 Views
- 8 replies
- 2 kudos
Resolved! When will DB release runtime with Scala 2.13
When will DB release runtime with Scala 2.13? Thx
- 840 Views
- 8 replies
- 2 kudos
Latest Reply
- 2 kudos
Spark 4 is coming and it's only going to support Scala 2.13, so a Databricks Runtime with Spark 3.5.x and Scala 2.13 should be released soonish.
- 2 kudos
- 86 Views
- 3 replies
- 0 kudos
Resolved! Error when executing an INSERT statement on an External Postgres table from Databricks SQL Editor
Hi,This is the context of my issue:I have an AWS RDS Postgres database instance setup. I have also set up a Postgres CONNECTION in Databricks and can view the Postgres tables under a newly created FOREIGN CATALOG in Databricks Unity Catalog.Using the...
- 86 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @pankj0510, DML for tables is blocked from Databricks SQL, you can only read from DBSQL. I think you can set up a JDBC URL to the Postgres database and use Spark/Pandas DataFrame write methods to insert data
- 0 kudos
- 28 Views
- 0 replies
- 0 kudos
how to use azure one lake in aws databricks unity catalog
i'm trying to connect azure one lake in aws databricks unity catalog but i'm not able to storage credential, since it's currently allowing s3 location only but in hive catalog i'm able to connect to one lake but not in unity.
- 28 Views
- 0 replies
- 0 kudos
- 132 Views
- 1 replies
- 0 kudos
Resolved! Custom VPC Subranges for New GCP Databricks Deployment
What Pods and Services subranges would you recommend for a /22 subnet for a custom VPC for a new GCP Databricks deployment in the GCE era?
- 132 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
The secondary ranges are there to support legacy GKE clusters. While required in the UI, they can be empty in terraform (per a source) for new deployments as clusters are GCE now. (There is a green GCE next to the cluster name.) When observing the ...
- 0 kudos
- 5407 Views
- 2 replies
- 4 kudos
Merge - Deep Dive
In this blog we are going beyond the basics to explore the internals of Databricks Merge into. By the end of this article, you will learn: How the merge command works.How running MERGE using Photon + Liquid clustering the target table can supercharg...
- 5407 Views
- 2 replies
- 4 kudos
Latest Reply
- 4 kudos
Thanks for the post. I don't understand this statement: Although classic warehouses support certain Photon operations, they don’t fully Photonize Merge. Also for this scenario the tables don’t have liquid clustering enabled. Why is not fully Photoniz...
- 4 kudos
- 125 Views
- 1 replies
- 0 kudos
Text alignment in databricks dashboard markdown
Hi All,How can I align the text inside the Dashboard markdown to the middle?Is there an option to do this?Thanks,Gal
- 125 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello @Gal_Sb! Databricks markdown does not support text alignment, and HTML/CSS do not work for this purpose in Databricks dashboards. You can try formatting options like headers or spacing adjustments. I'll also check with the team to explore possi...
- 0 kudos
- 554 Views
- 8 replies
- 4 kudos
- 554 Views
- 8 replies
- 4 kudos
Latest Reply
- 4 kudos
@Sergecom The engineering team worked on this and I believe it is now resolved for the newer runtimes. I confirmed it nows works as expected on 16.2. Can you confirm if it works on your end?
- 4 kudos
- 19 Views
- 0 replies
- 0 kudos
Join Us for Data Intelligence Day in Milan!
Tuesday, April 1, 2025, 9:00 AM-5:00 PM 5:30 PM Onwards Enterprise Hotel, Corso Sempione, 91, Milano 20149 Secure your spot today! Register Here! We invite you to join us for a half-day event where you can dive deeper into the approaches you can u...
- 19 Views
- 0 replies
- 0 kudos
- 19 Views
- 0 replies
- 0 kudos
Join Us for Data Intelligence Day in Dublin!
Tuesday, April 8, 2025, 8:30 AM-5:30 PM GMT 5:30 PM Onwards KPMG Office, 2 Harbourmaster Place, Dublin 1, D01 X5P3 Secure your spot today! Register Here! Join us for a half-day exploration into approaches you can use to deliver more success with yo...
- 19 Views
- 0 replies
- 0 kudos
Announcements
| User | Count |
|---|---|
| 106 | |
| 43 | |
| 22 | |
| 16 | |
| 14 |
Latest from our Blog
How to Assess and Adopt New Technologies Using Data-Driven Proof of Concept
In this article, I share a #data-driven method for assessing and adopting new technologies and steps for creating or evaluating a strong proposal. This includes using statistical measures to evaluate ...
[PARTNER BLOG] Introducing Databricks Native Master Data Management (MDM) - Entity Resolution
LakeFusion’s MDM solution delivers a single source of truth by leveraging advanced entity resolution and deduplication algorithms. By unifying fragmented data across systems, we ensure accurate, consi...
💡 ML Training Tip Of The Week #5: Enable fine-grained data frequency in AutoML forecast
Customers with high-frequency time series data often need forecasts at a finer granularity based on their data sources. For example, an energy supply company may receive energy demand data every 15 mi...
💡 ML Training Tip Of The Week #4: Speed up your ML workload with one toggle
Machine learning workload can be time-consuming due to training data processing such as ETL and feature engineering and iterative model training steps. Recent ML Runtime releases have made both faster...
