Unity Catalog Pandas on Spark Limitation

- - Certifications
- - Learning Paths

- - Community Discussions
- - Get Started Discussions

- - Get Started Resources
- - Events
- - Product Platform Updates
- - Support FAQs
- - Technical Blog
- - Get Started Guides
- - Knowledge Sharing Hub
- - Announcements
- - DatabricksTV

- - Private Groups
- - Skills@Scale

- - Databricks Community Champions
- - Khoros Community Forums Support (Not for Databricks Product Questions)
- - Databricks Community Code of Conduct

Administration & Architecture

Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.

According to Databricks UC Documentation, below are the some of the limitations on Shared Mode Cluster.

1. In Databricks Runtime 13.3 LTS and above, Python scalar UDFs and Pandas UDFs are supported. Other Python UDFs, including UDAFs, UDTFs, and Pandas on Spark are not supported.

Why is Pandas on Spark not supported in "Shared" Access mode clusters when they are UC enabled?

Is there a feature getting added to support this soon?