cancel
Showing results for 
Search instead for 
Did you mean: 
Knowledge Sharing Hub
Dive into a collaborative space where members like YOU can exchange knowledge, tips, and best practices. Join the conversation today and unlock a wealth of collective wisdom to enhance your experience and drive success.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Yogic24
by New Contributor III
  • 380 Views
  • 1 replies
  • 3 kudos

Why we are using shared variables into spark?

Hello Databricks Community!I am excited to share my first blog post with you all. This is a small and basic introduction to the concept of shared variables in Apache Spark. I hope this post will help those who are new to Spark understand why shared ...

  • 380 Views
  • 1 replies
  • 3 kudos
Latest Reply
Rishabh_Tiwari
Community Manager
  • 3 kudos

Hi @Yogic24 , Welcome to the Databricks Community! Thank you for sharing your first blog post with us, I am sure it will help our community members. Thank you for your contribution and support!   Thanks,Rishabh

  • 3 kudos
Yogic24
by New Contributor III
  • 787 Views
  • 2 replies
  • 5 kudos

knowledge sharing

Hello Databricks Community,I hope this message finds you well.My name is Yogita, and I am excited to announce that I have started sharing knowledge and insights on various Spark scenarios and theoretical questions on linkdin platform. My aim is to cr...

  • 787 Views
  • 2 replies
  • 5 kudos
Latest Reply
Sujitha
Community Manager
  • 5 kudos

Hi @Yogic24 , that's fantastic! Please share your blogs in this space so fellow data practitioners can benefit.

  • 5 kudos
1 More Replies
Ajay-Pandey
by Esteemed Contributor III
  • 248 Views
  • 1 replies
  • 1 kudos

Enhanced Search in Schema Browser

More easily find models and tables in your organization with our enhanced Schema Browser search experience. Functions and volumes support are coming soon. Opt-in to use this feature by visiting Settings > User > Developer settings. 

AjayPandey_0-1719456156051.png
Knowledge Sharing Hub
schema
Unity Catalog
  • 248 Views
  • 1 replies
  • 1 kudos
Latest Reply
Rishabh_Tiwari
Community Manager
  • 1 kudos

Thank you for sharing this update with the community! @Ajay-Pandey  Thanks for keeping the community informed.

  • 1 kudos
Ajay-Pandey
by Esteemed Contributor III
  • 314 Views
  • 1 replies
  • 2 kudos

Can monitor permission for SQL Warehouses now in Public Preview

Can monitor permission allows users to monitor SQL Warehouses, including query history and query profiles. 

AjayPandey_0-1719456054667.png
Knowledge Sharing Hub
ACL
monitoring
  • 314 Views
  • 1 replies
  • 2 kudos
Latest Reply
Rishabh_Tiwari
Community Manager
  • 2 kudos

Hi @Ajay-Pandey , Thank you for sharing this. I am sure it will help community members. We appreciate your participation.  Thanks, Rishabh

  • 2 kudos
Ajay-Pandey
by Esteemed Contributor III
  • 537 Views
  • 1 replies
  • 2 kudos

Bulk move and delete available in filebrowser

You can now perform bulk move and delete on your filebrowser assets. Select multiple assets and right-click or use the new actions toolbar.

  • 537 Views
  • 1 replies
  • 2 kudos
Latest Reply
Rishabh_Tiwari
Community Manager
  • 2 kudos

Thank you for sharing the quick tricks @Ajay-Pandey I am sure, it will help other community members.

  • 2 kudos
Ajay-Pandey
by Esteemed Contributor III
  • 296 Views
  • 1 replies
  • 2 kudos

Caching and data freshness

Dashboards maintain a 24-hour result cache to optimize initial loading times, operating on a best-effort basis. This means that while the system always attempts to use historical query results linked to dashboard credentials to enhance performance, t...

Knowledge Sharing Hub
caching
warehouse
  • 296 Views
  • 1 replies
  • 2 kudos
Latest Reply
Rishabh_Tiwari
Community Manager
  • 2 kudos

Thank you for providing this valuable update on caching and data freshness @Ajay-Pandey . It's great to see how these features are designed to optimize performance and ensure data accuracy. We appreciate your detailed explanation and your efforts to ...

  • 2 kudos
Ajay-Pandey
by Esteemed Contributor III
  • 210 Views
  • 1 replies
  • 2 kudos

Accelerating discovery on Unity Catalog with a revamped Catalog Explore

Discover favorite and recent UC assets in Quick Access. You'll experience a simplified navigation with the gear icon (top left) for compute, storage, credentials, connections, DBFS, and managements features. Delta Sharing, Clean Rooms, and External D...

  • 210 Views
  • 1 replies
  • 2 kudos
Latest Reply
Rishabh_Tiwari
Community Manager
  • 2 kudos

Thank you for sharing this update on the Unity Catalog! @Ajay-Pandey Appreciate the detailed overview!

  • 2 kudos
alysson_souza
by New Contributor III
  • 570 Views
  • 1 replies
  • 3 kudos

Configuring DNS resolution for Private Databricks Workspaces (AWS)

Intro For customers on the E2 Platform, Databricks has a feature that allows them to use AWS PrivateLink to provision secure private workspaces by creating VPC endpoints to both the front-end and back-end interfaces of the Databricks infrastructure. ...

AWS Inbound DNS Endpoints for Workspaces - Copy of Page 1.png alysson_souza_1-1718062496966.png AWS Inbound DNS Endpoints for Workspaces - Copy of Page 1.png
  • 570 Views
  • 1 replies
  • 3 kudos
Latest Reply
Sujitha
Community Manager
  • 3 kudos

@alysson_souza love it! Thank you for sharing

  • 3 kudos
Harun
by Honored Contributor
  • 811 Views
  • 1 replies
  • 1 kudos

Optimizing Costs in Databricks by Dynamically Choosing Cluster Sizes

Databricks is a popular unified data analytics platform known for its powerful data processing capabilities and seamless integration with Apache Spark. However, managing and optimizing costs in Databricks can be challenging, especially when it comes ...

  • 811 Views
  • 1 replies
  • 1 kudos
Latest Reply
Sujitha
Community Manager
  • 1 kudos

@Harun This is amazing! Thank you for sharing

  • 1 kudos
SashankKotta
by New Contributor III
  • 457 Views
  • 1 replies
  • 2 kudos

Library Management via Custom Compute Policies and ADF Job Triggering

This guide is intended for those looking to install libraries on a cluster using a Custom Compute Policy and trigger Databricks jobs from an Azure Data Factory (ADF) linked service. While many users rely on init scripts for library installation, it i...

Screenshot 2024-06-16 at 12.34.09 PM.png Screenshot 2024-06-16 at 12.38.33 PM.png
  • 457 Views
  • 1 replies
  • 2 kudos
Latest Reply
Sujitha
Community Manager
  • 2 kudos

@SashankKotta Thank you for sharing!

  • 2 kudos
Harun
by Honored Contributor
  • 788 Views
  • 1 replies
  • 2 kudos

Comprehensive Guide to Databricks Optimization: Z-Order, Data Compaction, and Liquid Clustering

Optimizing data storage and access is crucial for enhancing the performance of data processing systems. In Databricks, several optimization techniques can significantly improve query performance and reduce costs: Z-Order Optimize, Optimize Compaction...

  • 788 Views
  • 1 replies
  • 2 kudos
Latest Reply
Sujitha
Community Manager
  • 2 kudos

Very insightful @Harun. Thank you for sharing this with us!

  • 2 kudos
SashankKotta
by New Contributor III
  • 629 Views
  • 0 replies
  • 1 kudos

CICD for databricks workflow jobs

This post is to set up Databricks workflow jobs as a CI/CD. Below are the two essential components needed for a complete CI/CD setup of workflow jobs. Databricks Asset Bundles(DABs)AzureDevOps pipeline. Databricks Asset Bundle ( From local terminal )...

Screenshot 2024-06-14 at 4.27.27 PM.png Screenshot 2024-06-14 at 3.59.46 PM.png Screenshot 2024-06-14 at 4.19.59 PM.png
  • 629 Views
  • 0 replies
  • 1 kudos
Rishabh-Pandey
by Honored Contributor II
  • 401 Views
  • 0 replies
  • 0 kudos

The Importance of Databricks in Today's Data Market

  Unlocking the Power of Data with Databricks In the rapidly evolving landscape of data and analytics, Databricks has emerged as a transformative force, reshaping how organizations handle big data, data engineering, and machine learning. As we naviga...

  • 401 Views
  • 0 replies
  • 0 kudos
RamkannanA
by New Contributor II
  • 1035 Views
  • 1 replies
  • 2 kudos

Resolved! RamK - Certification Update

Hi Team,My name is Ram based out of Singapore. I am new to this Community . Recently I have completed my certification in Databricks starting from Data Analyst , Data Engineering and Gen AI. Looking forward to get connected in serving the Data and AI...

RamkannanA_1-1718261550063.png
  • 1035 Views
  • 1 replies
  • 2 kudos
Latest Reply
Rishabh-Pandey
Honored Contributor II
  • 2 kudos

Happy to see you here .

  • 2 kudos