cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Some key notes from today's Databricks Azure Roadmap Q4 event: - Faster merge and updates with Photon + Deletion vectors in Q3,- Unity Catalog - s...

Hubert-Dudek
Esteemed Contributor III

Some key notes from today's Databricks Azure Roadmap Q4 event:

- Faster merge and updates with Photon + Deletion vectors in Q3,

- Unity Catalog - system tables, including system table for lineage,

- ML features and models under Unity Catalog governance,

- Auto tuning for managed tables in Unity Catalog

- Delta sharing Now GA,

- Databricks marketplace (private preview in Q4),

- The private link between Databricks Control Plane and corporate network,

- HIPAA and other compliance GA,

- Customer managed keys from databricks services and managed disks coming soon,

- Cost management in databricks console, charts, budgets, etc. (in development),

- Serverless SQL (public preview),

- Materialized views (in development),

- Automatic workload optimization (so small queries don't need to wait for big ones to finish),

- Full REST API, including SQL querying (private preview)

- Create an external table in any database of your choice and query it from databricks,

- Pythons UDFS so your python code can be registered directly as a permanent function in the catalog,

- SQL queries in workflows,

- Run SQL query as the service account,

- Dashboards in git,

- Enhanced workflow logic, including True/Falce, Fail/Success steps,

- Workflow trigger when new data arrive (private preview)

- ML autoscaling endpoint to serve ML models in production (private preview)

- Delta live tables integrated with ML,

- The feature store can be combined with Delta Live Tables (private preview)

- MLOps templates (private preview)

- Share dashboards to non-databricks users,

- Include dashboards on the web page,

- The e-nzyme in development (optimal update techniques)

- SCT Type 2 in preview,

- Serverless jobs (private preview in Q3),

- Enhanced monitoring and alerts, including visuals and Slack and other hooks,

- Primary/foreign keys in public preview, 

- Jupiter support in notebooks,

- Repos enhancements (rebase etc.)

- Workspace browser to see all objects (aka file browser) (in development)

- VS code extensions Private Preview Q3

5 REPLIES 5

Pat
Honored Contributor III

Thanks, for sharing this.

Nothing about Delta Live Tables availability in Unity Catalog ?

Hubert-Dudek
Esteemed Contributor III

Sadly I have no idea about that topic

Kaniz
Community Manager
Community Manager

Hi @Hubert Dudek​ , Thank you for sharing such a fantastic post with our community. You're such an inspiration!

Also, Please don't forget to click on the "Select As Best" button if you like my comment.

Hubert-Dudek
Esteemed Contributor III

You are awesome!

Thank you @Hubert Dudek​ !

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!