cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 
Help
Knowledge Sharing Hub
Dive into a collaborative space where members like YOU can exchange knowledge, tips, and best practices. Join the conversation today and unlock a wealth of collective wisdom to enhance your experience and drive success.
cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
SumitSingh
by • Contributor
  • 3594 Views
  • 7 replies
  • 11 kudos

From Associate to Professional: My Learning Plan to ace all Databricks Data Engineer Certifications

In today’s data-driven world, the role of a data engineer is critical in designing and maintaining the infrastructure that allows for the efficient collection, storage, and analysis of large volumes of data. Databricks certifications holds significan...

SumitSingh_0-1721402402230.png SumitSingh_1-1721402448677.png SumitSingh_2-1721402469214.png
Knowledge Sharing Hub
Reply
  • 3594 Views
  • 7 replies
  • 11 kudos
View Replies
Latest Reply
sandeepmankikar
New Contributor III
  • 11 kudos

As an additional tip for those working towards both the Associate and Professional certifications, I recommend avoiding a long gap between the two exams to maintain your momentum. If possible, try to schedule them back-to-back with just a few days in...

  • 11 kudos
6 More Replies
Yogic24
by • Valued Contributor II
  • 4012 Views
  • 0 replies
  • 1 kudos

Reading modes in spark

Hello Everyone, 3 type of reading mode into pyspark:1.Permissive Mode : its a default mode. if spark is not able to parse it due to dattype mismatch then make it as null without impacting other result, i.e only issue record are null others are fine.2...

Knowledge Sharing Hub
Reply
  • 4012 Views
  • 0 replies
  • 1 kudos
Ajay-Pandey
by • Esteemed Contributor III
  • 724 Views
  • 0 replies
  • 0 kudos

Lakehouse Monitoring GA: Profile, diagnose, and enforce data quality

Lakehouse Monitoring provides automated profiling and a dashboard that visualizes trends and anomalies over time. Track key metrics such as data volume, percent nulls, numerical distribution changes, and categorical distribution. For inference tables...

AjayPandey_0-1720668179744.png
Knowledge Sharing Hub
Lakehouse
monitoring
Unity Catalog
Reply
  • 724 Views
  • 0 replies
  • 0 kudos
Ajay-Pandey
by • Esteemed Contributor III
  • 994 Views
  • 2 replies
  • 3 kudos

Resolved! Observability for materialized views and streaming tables

You can now view a materialized view/streaming table's refresh information and schedules from the Catalog Explorer page. 

AjayPandey_0-1720149404659.png
Knowledge Sharing Hub
Reply
  • 994 Views
  • 2 replies
  • 3 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 3 kudos

Thank you for sharing this valuable information with the community! @Ajay-Pandey This will surely help many users manage their data workflows more efficiently. We appreciate your contribution and encourage more insights like this. 

  • 3 kudos
1 More Replies
Ajay-Pandey
by • Esteemed Contributor III
  • 2881 Views
  • 1 replies
  • 4 kudos

Best Practices for Utilizing Databricks Asset Bundles

Best practices for utilizing Databricks Asset Bundles effectively streamline your data and AI projects, ensuring they are robust, scalable, and maintainable. Here are some key practices to consider:1. Version Control: Always maintain a versioned hist...

Knowledge Sharing Hub
data engineering
databricks bundle
databricks cicd
Reply
  • 2881 Views
  • 1 replies
  • 4 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 4 kudos

Thank you for sharing these best practices. @Ajay-Pandey We appreciate your contribution to helping the community members streamline their projects effectively. 

  • 4 kudos
Rishabh-Pandey
by • Esteemed Contributor
  • 1693 Views
  • 2 replies
  • 4 kudos

Resolved! DLT pipeline development made simple with notebooks

We’re just a couple weeks removed from the biggest Data + AI Summit in history, where we introduced Databricks LakeFlow, a unified, intelligent solution for data engineering. While we are hard at work evolving DLT and Workflows into the new unified s...

Knowledge Sharing Hub
Reply
  • 1693 Views
  • 2 replies
  • 4 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 4 kudos

Hi @Rishabh-Pandey  Thank you for sharing this insightful update! We're thrilled to see your enthusiasm for the latest enhancements to the DLT development experience. Please keep sharing your experiences and insights with the community. Thanks,Rishab...

  • 4 kudos
1 More Replies
Ajay-Pandey
by • Esteemed Contributor III
  • 1667 Views
  • 1 replies
  • 1 kudos

Resolved! Databricks SQL fixes

Added support for customizing scatter point size and opacity in AI/BI dashboards.Improved interactivity in displaying tooltips when hovering over pie, scatter, and heatmap charts with many data points.AI/BI dashboard widgets created by Databricks Ass...

Knowledge Sharing Hub
Databricks
sql
sqlwarehouse
Reply
  • 1667 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 1 kudos

Thank you for sharing the updates on fixes @Ajay-Pandey . 

  • 1 kudos
Yogic24
by • Valued Contributor II
  • 4238 Views
  • 1 replies
  • 3 kudos

What is Accumulators and Broadcast Variables

1) Accumulators: Accumulators are used to implement counters and sum in Spark applications.Accumulators allow you to aggregate values from tasks running on worker nodes back to the driver program. They provide a way for tasks to incrementally update ...

Knowledge Sharing Hub
Reply
  • 4238 Views
  • 1 replies
  • 3 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 3 kudos

Thank you for sharing this @Yogic24 . I am sure it will help other community members.

  • 3 kudos
Yogic24
by • Valued Contributor II
  • 1390 Views
  • 1 replies
  • 3 kudos

Why we are using shared variables into spark?

Hello Databricks Community!I am excited to share my first blog post with you all. This is a small and basic introduction to the concept of shared variables in Apache Spark. I hope this post will help those who are new to Spark understand why shared ...

Knowledge Sharing Hub
data engineer
Reply
  • 1390 Views
  • 1 replies
  • 3 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 3 kudos

Hi @Yogic24 , Welcome to the Databricks Community! Thank you for sharing your first blog post with us, I am sure it will help our community members. Thank you for your contribution and support!   Thanks,Rishabh

  • 3 kudos
Yogic24
by • Valued Contributor II
  • 1395 Views
  • 2 replies
  • 5 kudos

knowledge sharing

Hello Databricks Community,I hope this message finds you well.My name is Yogita, and I am excited to announce that I have started sharing knowledge and insights on various Spark scenarios and theoretical questions on linkdin platform. My aim is to cr...

Knowledge Sharing Hub
Reply
  • 1395 Views
  • 2 replies
  • 5 kudos
View Replies
Latest Reply
Sujitha
Databricks Employee
  • 5 kudos

Hi @Yogic24 , that's fantastic! Please share your blogs in this space so fellow data practitioners can benefit.

  • 5 kudos
1 More Replies
Ajay-Pandey
by • Esteemed Contributor III
  • 1549 Views
  • 1 replies
  • 1 kudos

Enhanced Search in Schema Browser

More easily find models and tables in your organization with our enhanced Schema Browser search experience. Functions and volumes support are coming soon. Opt-in to use this feature by visiting Settings > User > Developer settings. 

AjayPandey_0-1719456156051.png
Knowledge Sharing Hub
schema
Unity Catalog
Reply
  • 1549 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 1 kudos

Thank you for sharing this update with the community! @Ajay-Pandey  Thanks for keeping the community informed.

  • 1 kudos
Ajay-Pandey
by • Esteemed Contributor III
  • 1342 Views
  • 1 replies
  • 2 kudos

Can monitor permission for SQL Warehouses now in Public Preview

Can monitor permission allows users to monitor SQL Warehouses, including query history and query profiles. 

AjayPandey_0-1719456054667.png
Knowledge Sharing Hub
ACL
monitoring
Reply
  • 1342 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 2 kudos

Hi @Ajay-Pandey , Thank you for sharing this. I am sure it will help community members. We appreciate your participation.  Thanks, Rishabh

  • 2 kudos
Ajay-Pandey
by • Esteemed Contributor III
  • 933 Views
  • 1 replies
  • 2 kudos

Bulk move and delete available in filebrowser

You can now perform bulk move and delete on your filebrowser assets. Select multiple assets and right-click or use the new actions toolbar.

Knowledge Sharing Hub
Reply
  • 933 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 2 kudos

Thank you for sharing the quick tricks @Ajay-Pandey I am sure, it will help other community members.

  • 2 kudos

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels
li.common.scroll-to.top