cancel
Showing results for 
Search instead for 
Did you mean: 
Knowledge Sharing Hub
Dive into a collaborative space where members like YOU can exchange knowledge, tips, and best practices. Join the conversation today and unlock a wealth of collective wisdom to enhance your experience and drive success.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

SumitSingh
by Contributor
  • 2369 Views
  • 6 replies
  • 8 kudos

From Associate to Professional: My Learning Plan to ace all Databricks Data Engineer Certifications

In today’s data-driven world, the role of a data engineer is critical in designing and maintaining the infrastructure that allows for the efficient collection, storage, and analysis of large volumes of data. Databricks certifications holds significan...

SumitSingh_0-1721402402230.png SumitSingh_1-1721402448677.png SumitSingh_2-1721402469214.png
  • 2369 Views
  • 6 replies
  • 8 kudos
Latest Reply
jem
New Contributor II
  • 8 kudos

This is great! I have worked with Databricks for almost three years and has decided to pursue the Databricks Engineer Professional certification. This will certainly help setting up an effective plan.

  • 8 kudos
5 More Replies
Ajay-Pandey
by Esteemed Contributor III
  • 3371 Views
  • 6 replies
  • 7 kudos

🚀 Databricks Custom Apps! 🚀

Whether you're a data scientist or a sales executive, Databricks is making it easier than ever to build, host, and share secure data applications. With our platform, you can now run any Python code on serverless compute, share it with non-technical c...

  • 3371 Views
  • 6 replies
  • 7 kudos
Latest Reply
PiotrU
Contributor II
  • 7 kudos

Can we somehow play with hosting, and expose this app outside?

  • 7 kudos
5 More Replies
Thusharr
by New Contributor II
  • 1174 Views
  • 5 replies
  • 1 kudos

Writing append blob files to unity catalog volum

The workspace is assigned to unity catalog, and all the access to the ADLS Gen2 is now handled via unity catalog only, means no SPN, no connection string, access keys etc. I have to create append blob files in a volume, Is this is possible in a works...

  • 1174 Views
  • 5 replies
  • 1 kudos
Latest Reply
Witold
Honored Contributor
  • 1 kudos

Now I got your point. No, you can't create Append Blob files directly in Volumes, as this is a native Azure functionality. A volume is basically just an abstraction over a native storage.You will still need to use libraries like azure-storage-blob wi...

  • 1 kudos
4 More Replies
Thusharr
by New Contributor II
  • 2546 Views
  • 1 replies
  • 0 kudos

Unity cataog

Disaster recovery is possible in Unity catalog now?Means, for data level, we have enabled with geo redundancy, what about the objects, permissions, an other components in Unity catalog ? Can we restore the unity catalog metadata in another region ?

  • 2546 Views
  • 1 replies
  • 0 kudos
Latest Reply
gchandra
Databricks Employee
  • 0 kudos

Official product release in development will be available as PrPr in a few months. 

  • 0 kudos
arjungoel1995
by New Contributor
  • 1422 Views
  • 2 replies
  • 0 kudos

Resolved! Standardized Framework to update Databricks job definition using CI/CD

Hi Databricks support, I am looking for a standardized Databricks framework to update job definition using DevOps from non-production till it get productionized. Our current process of updating the Databricks job definition is as follows:In our sourc...

  • 1422 Views
  • 2 replies
  • 0 kudos
Latest Reply
nicole_lu_PM
Databricks Employee
  • 0 kudos

Hi from the Git folders/Repos PM: DAB is the way to go, and we are working on an integration to author DABs directly in the workspace.  Here's a DAIS talk where the DAB PM and I demo'ed some recommendations for source controlling jobs: https://www.da...

  • 0 kudos
1 More Replies
ThierryBa
by New Contributor III
  • 3196 Views
  • 3 replies
  • 5 kudos

Build & Refresh a Calendar Dates Table

IntroductionMaintaining accurate and up-to-date calendar date tables is crucial for reliable reporting, yet manual updates can be time-consuming and prone to error. This fundamental component serves as the backbone for date-based analysis, enabling a...

ThierryBa_0-1726544617449.png ThierryBa_1-1726544617449.png ThierryBa_2-1726544617450.png
  • 3196 Views
  • 3 replies
  • 5 kudos
Latest Reply
may-tun
New Contributor II
  • 5 kudos

Nice article, very informative! 

  • 5 kudos
2 More Replies
AhmedAlnaqa
by Contributor
  • 4348 Views
  • 3 replies
  • 1 kudos

Reading Excel files folder

Dears,One of the tasks needed by DE is to ingest data from files, for example, Excel file.Thanks for OnerFusion-AI for the below thread that give us the steps of reading from one file https://community.databricks.com/t5/get-started-discussions/how-to...

  • 4348 Views
  • 3 replies
  • 1 kudos
Latest Reply
maddy08
New Contributor II
  • 1 kudos

Hi @AhmedAlnaqa ,Can we read from ADLS location too by using abfss ?Thanks

  • 1 kudos
2 More Replies
Ajay-Pandey
by Esteemed Contributor III
  • 647 Views
  • 1 replies
  • 3 kudos

Notification destination API

You can now create, delete, and update notification destinations through the Databricks API.The notification destinations API lets you programmatically manage a workspace's notification destinations. Notification destinations are used to send notific...

Knowledge Sharing Hub
databricks api
notificatio api
  • 647 Views
  • 1 replies
  • 3 kudos
Latest Reply
Anushree_Tatode
Contributor III
  • 3 kudos

Hi,Thank you for sharing this Ajay. We appreciate you keeping the community informed!Thanks,Anushree

  • 3 kudos
Ajay-Pandey
by Esteemed Contributor III
  • 1593 Views
  • 2 replies
  • 4 kudos

Onboarding your new Databricks AI/BI Genie

The integration of AI and BI into the modern data stack has been a game-changer for businesses seeking to leverage data-driven insights. Databricks, a leader in this innovative frontier, has introduced the AI/BI Genie, a tool designed to democratize ...

Knowledge Sharing Hub
AI
assistance
bi
data engineering
Databricks
  • 1593 Views
  • 2 replies
  • 4 kudos
Latest Reply
Anushree_Tatode
Contributor III
  • 4 kudos

Hi,The AI/BI Genie is a fantastic innovation! By enabling natural language queries and learning from user interactions, it makes data analytics more accessible and insightful. It’s a powerful tool for businesses looking to enhance their data-driven d...

  • 4 kudos
1 More Replies
Rishabh-Pandey
by Esteemed Contributor
  • 1027 Views
  • 1 replies
  • 2 kudos

The Importance of Databricks in Today's Data Market

  Unlocking the Power of Data with Databricks In the rapidly evolving landscape of data and analytics, Databricks has emerged as a transformative force, reshaping how organizations handle big data, data engineering, and machine learning. As we naviga...

  • 1027 Views
  • 1 replies
  • 2 kudos
Latest Reply
Anushree_Tatode
Contributor III
  • 2 kudos

Hi @Rishabh-Pandey,Thank you for sharing this! Databricks is really making a difference with its unified platform and powerful features. Excited to see how it will continue to shape the future of data! Thanks,

  • 2 kudos
Rishabh-Pandey
by Esteemed Contributor
  • 6208 Views
  • 1 replies
  • 1 kudos

Databricks on Databricks: Kicking off the Journey to Governance with Unity Catalog

In the world of data engineering and analytics, governance is paramount. Databricks has taken a significant step forward in this arena with the introduction of Unity Catalog, a unified governance solution for all data and AI assets. This journey to e...

  • 6208 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anushree_Tatode
Contributor III
  • 1 kudos

Hi Rishabh ,Unity Catalog is a major advancement in data governance, enhancing how we manage and secure data and AI assets in Databricks. Exciting to see these improvements!Thanks,Anushree 

  • 1 kudos
Brahmareddy
by Honored Contributor
  • 737 Views
  • 1 replies
  • 2 kudos

How to detect the Risks in Claims Data Using Databricks and PySpark

As a data engineer with experience in Databricks and other data engineering tools, I know that processing claims data and detecting risks early can really help in insurance claims processing. In this article, I’ll show you how to use Databricks and P...

  • 737 Views
  • 1 replies
  • 2 kudos
Latest Reply
Anushree_Tatode
Contributor III
  • 2 kudos

Hi, Thanks for sharing this! Kudos for breaking it down so clearly. I’m sure, it will help other community members. Thanks,Anushree

  • 2 kudos
Ajay-Pandey
by Esteemed Contributor III
  • 393 Views
  • 0 replies
  • 2 kudos

September 2024 - Databricks SQL fixes

Added customization options for number formats in dashboard widgets, including currencies and percentages.The width of the Values column in pivot tables now auto-adjusts based on the width of the cell value names.The time displayed in dashboard widge...

  • 393 Views
  • 0 replies
  • 2 kudos
wheersink
by New Contributor
  • 440 Views
  • 1 replies
  • 0 kudos

SQL code for appending a notebook result into an existing database table

I am attempting to append the results from a notebook query results table into an existing databricks database table.  By chance would someone share an example of the sql code with me?   

  • 440 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 0 kudos

Hi @wheersink ,So let's say you created following table with some sample values.%sql CREATE TABLE dev.default.employee ( id INT, name STRING, age INT, department STRING ); INSERT INTO dev.default.employee VALUES (1, 'John Doe', 30, 'Financ...

  • 0 kudos
Ajay-Pandey
by Esteemed Contributor III
  • 1196 Views
  • 1 replies
  • 2 kudos

🚀 Boost Your Data Pipelines with Dynamic, Data-Driven Databricks Workflows (For Each Task)! 💡

Unlock the power of the For Each task in Databricks to seamlessly iterate over collections—whether it's a list of table names or any value—and dynamically run tasks with specific parameter values. This powerful feature lets you automate repetitive pr...

AjayPandey_0-1724212332991.gif
Knowledge Sharing Hub
automation
bigdata
Databricks
dataengineering
Workfow
  • 1196 Views
  • 1 replies
  • 2 kudos
Latest Reply
Rishabh-Pandey
Esteemed Contributor
  • 2 kudos

Thanks for sharing @Ajay-Pandey 

  • 2 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group