cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Knowledge Sharing Hub
Dive into a collaborative space where members like YOU can exchange knowledge, tips, and best practices. Join the conversation today and unlock a wealth of collective wisdom to enhance your experience and drive success.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
nihalataskeen
by • New Contributor III
  • 458 Views
  • 3 replies
  • 4 kudos

Introducing Databricks LakeFlow: A New Solution for Data Engineering

Hello Everyone!!!I hope everyone knows about the exciting new launch from Databricks: LakeFlow. This innovative tool makes data engineering simpler and more efficient. Let’s understand what LakeFlow offers and how it can help data teams. Let’s unders...

nihalataskeen_0-1721208704206.png nihalataskeen_1-1721208784471.png nihalataskeen_2-1721208855689.png
Knowledge Sharing Hub
Reply
  • 458 Views
  • 3 replies
  • 4 kudos
View Replies
Latest Reply
SharathE
New Contributor III
  • 4 kudos

Hi @nihalataskeen Thankyou and I have signed up in the waitlist. Any idea when can we might get access to this??

  • 4 kudos
2 More Replies
Ajay-Pandey
by • Esteemed Contributor III
  • 96 Views
  • 0 replies
  • 0 kudos

Databricks User interface updates

AI/BI dashboards:Improvements:The assistant can now help you edit chart axis titles and control the visibility of chart data labels.You can now assign multiple measures to pivot tables.Added more relative datetime options, such as Last 10 seconds and...

Knowledge Sharing Hub
Reply
  • 96 Views
  • 0 replies
  • 0 kudos
Mot
by • New Contributor II
  • 147 Views
  • 3 replies
  • 0 kudos

Resolved! DevOps CICD for Databricks Schema

Does anyone know how to do DevOps CICD pipeline to deploy changes to a Databricks Catalog? I've added a couple tables in DEV and need to deploy in Prod. Could'nt find any info on this anywhere. 

Knowledge Sharing Hub
Reply
  • 147 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
Mot
New Contributor II
  • 0 kudos

Thanks @Kaniz_Fatma , the links refer mostly deployments of workspace objects rather than changes in database objects, eg. a new schema, table, or new column added to a table etc. I'll keep looking... 

  • 0 kudos
2 More Replies
SumitSingh
by • New Contributor III
  • 64 Views
  • 0 replies
  • 0 kudos

From Associate to Professional: My Learning Plan to ace all Databricks Data Engineer Certifications

In today’s data-driven world, the role of a data engineer is critical in designing and maintaining the infrastructure that allows for the efficient collection, storage, and analysis of large volumes of data. Databricks certifications holds significan...

SumitSingh_0-1721402402230.png SumitSingh_1-1721402448677.png SumitSingh_2-1721402469214.png
Knowledge Sharing Hub
Reply
  • 64 Views
  • 0 replies
  • 0 kudos
Ajay-Pandey
by • Esteemed Contributor III
  • 132 Views
  • 1 replies
  • 0 kudos

Databricks Terraform Authentication Methods

Databricks, a unified data analytics platform, is widely recognized for its ability to process big data and run complex algorithms. When it comes to infrastructure as code, Terraform stands out as a tool that can manage Databricks resources efficient...

Knowledge Sharing Hub
Databricks
Terraform
Reply
  • 132 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 0 kudos

Hi @Ajay-Pandey , Thank you for sharing this comprehensive guide on managing Databricks resources with Terraform. The detailed explanation of the different authentication methods and examples is very helpful. I appreciate you taking the time to put t...

  • 0 kudos
Yogic24
by • New Contributor III
  • 202 Views
  • 0 replies
  • 0 kudos

Reading modes in spark

Hello Everyone, 3 type of reading mode into pyspark:1.Permissive Mode : its a default mode. if spark is not able to parse it due to dattype mismatch then make it as null without impacting other result, i.e only issue record are null others are fine.2...

Knowledge Sharing Hub
Reply
  • 202 Views
  • 0 replies
  • 0 kudos
Ajay-Pandey
by • Esteemed Contributor III
  • 132 Views
  • 0 replies
  • 0 kudos

Lakehouse Monitoring GA: Profile, diagnose, and enforce data quality

Lakehouse Monitoring provides automated profiling and a dashboard that visualizes trends and anomalies over time. Track key metrics such as data volume, percent nulls, numerical distribution changes, and categorical distribution. For inference tables...

AjayPandey_0-1720668179744.png
Knowledge Sharing Hub
Lakehouse
monitoring
Unity Catalog
Reply
  • 132 Views
  • 0 replies
  • 0 kudos
himoshi
by • New Contributor II
  • 572 Views
  • 2 replies
  • 0 kudos

Error code 403 - Invalid access to Org

I am trying to make a GET /api/2.1/jobs/list call in a Notebook to get a list of all jobs in my workspace but am unable to do so due to a 403 "Invalid access to Org" error message. I am using a new PAT and the endpoint is correct. I also have workspa...

Knowledge Sharing Hub
Reply
  • 572 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Walter_C
Honored Contributor
  • 0 kudos

Hello, this API you are running is to the same workspace where you are currently located? Are you using classic compute or user Serverless compute? In case it is serverless can you test it with classic? Also may I know on which region and which cloud...

  • 0 kudos
1 More Replies
Ajay-Pandey
by • Esteemed Contributor III
  • 207 Views
  • 2 replies
  • 2 kudos

Resolved! Observability for materialized views and streaming tables

You can now view a materialized view/streaming table's refresh information and schedules from the Catalog Explorer page. 

AjayPandey_0-1720149404659.png
Knowledge Sharing Hub
Reply
  • 207 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 2 kudos

Thank you for sharing this valuable information with the community! @Ajay-Pandey This will surely help many users manage their data workflows more efficiently. We appreciate your contribution and encourage more insights like this. 

  • 2 kudos
1 More Replies
Ajay-Pandey
by • Esteemed Contributor III
  • 328 Views
  • 1 replies
  • 3 kudos

Best Practices for Utilizing Databricks Asset Bundles

Best practices for utilizing Databricks Asset Bundles effectively streamline your data and AI projects, ensuring they are robust, scalable, and maintainable. Here are some key practices to consider:1. Version Control: Always maintain a versioned hist...

Knowledge Sharing Hub
data engineering
databricks bundle
databricks cicd
Reply
  • 328 Views
  • 1 replies
  • 3 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 3 kudos

Thank you for sharing these best practices. @Ajay-Pandey We appreciate your contribution to helping the community members streamline their projects effectively. 

  • 3 kudos
AhmedAlnaqa
by • New Contributor III
  • 366 Views
  • 1 replies
  • 1 kudos

Reading Excel files folder

Dears,One of the tasks needed by DE is to ingest data from files, for example, Excel file.Thanks for OnerFusion-AI for the below thread that give us the steps of reading from one file https://community.databricks.com/t5/get-started-discussions/how-to...

Knowledge Sharing Hub
Reply
  • 366 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 1 kudos

Hi @AhmedAlnaqa , Thank you for sharing this. I am sure it will help other community members. Thanks, Rishabh

  • 1 kudos
Rishabh-Pandey
by • Honored Contributor III
  • 428 Views
  • 2 replies
  • 2 kudos

Resolved! DLT pipeline development made simple with notebooks

We’re just a couple weeks removed from the biggest Data + AI Summit in history, where we introduced Databricks LakeFlow, a unified, intelligent solution for data engineering. While we are hard at work evolving DLT and Workflows into the new unified s...

Knowledge Sharing Hub
Reply
  • 428 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 2 kudos

Hi @Rishabh-Pandey  Thank you for sharing this insightful update! We're thrilled to see your enthusiasm for the latest enhancements to the DLT development experience. Please keep sharing your experiences and insights with the community. Thanks,Rishab...

  • 2 kudos
1 More Replies
Ajay-Pandey
by • Esteemed Contributor III
  • 664 Views
  • 1 replies
  • 1 kudos

Resolved! Databricks SQL fixes

Added support for customizing scatter point size and opacity in AI/BI dashboards.Improved interactivity in displaying tooltips when hovering over pie, scatter, and heatmap charts with many data points.AI/BI dashboard widgets created by Databricks Ass...

Knowledge Sharing Hub
Databricks
sql
sqlwarehouse
Reply
  • 664 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 1 kudos

Thank you for sharing the updates on fixes @Ajay-Pandey . 

  • 1 kudos
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!