Some key notes from today's Databricks Azure Roadmap Q4 event: - Faster merge and updates with Photon + Deletion vectors in Q3,- Unity Catalog - s...

Hubert-Dudek — Thu, 25 Aug 2022 16:18:06 GMT

Some key notes from today's Databricks Azure Roadmap Q4 event:

- Faster merge and updates with Photon + Deletion vectors in Q3,

- Unity Catalog - system tables, including system table for lineage,

- ML features and models under Unity Catalog governance,

- Auto tuning for managed tables in Unity Catalog

- Delta sharing Now GA,

- Databricks marketplace (private preview in Q4),

- The private link between Databricks Control Plane and corporate network,

- HIPAA and other compliance GA,

- Customer managed keys from databricks services and managed disks coming soon,

- Cost management in databricks console, charts, budgets, etc. (in development),

- Serverless SQL (public preview),

- Materialized views (in development),

- Automatic workload optimization (so small queries don't need to wait for big ones to finish),

- Full REST API, including SQL querying (private preview)

- Create an external table in any database of your choice and query it from databricks,

- Pythons UDFS so your python code can be registered directly as a permanent function in the catalog,

- SQL queries in workflows,

- Run SQL query as the service account,

- Dashboards in git,

- Enhanced workflow logic, including True/Falce, Fail/Success steps,

- Workflow trigger when new data arrive (private preview)

- ML autoscaling endpoint to serve ML models in production (private preview)

- Delta live tables integrated with ML,

- The feature store can be combined with Delta Live Tables (private preview)

- MLOps templates (private preview)

- Share dashboards to non-databricks users,

- Include dashboards on the web page,

- The e-nzyme in development (optimal update techniques)

- SCT Type 2 in preview,

- Serverless jobs (private preview in Q3),

- Enhanced monitoring and alerts, including visuals and Slack and other hooks,

- Primary/foreign keys in public preview,

- Jupiter support in notebooks,

- Repos enhancements (rebase etc.)

- Workspace browser to see all objects (aka file browser) (in development)

- VS code extensions Private Preview Q3

Re: Some key notes from today's Databricks Azure Roadmap Q4 event: - Faster merge and updates with Photon + Deletion vectors in Q3,- Unity Catalog - s...

Pat — Mon, 29 Aug 2022 11:57:24 GMT

Thanks, for sharing this.

Nothing about Delta Live Tables availability in Unity Catalog ?

Re: Some key notes from today's Databricks Azure Roadmap Q4 event: - Faster merge and updates with Photon + Deletion vectors in Q3,- Unity Catalog - s...

Hubert-Dudek — Mon, 29 Aug 2022 12:21:11 GMT

Sadly I have no idea about that topic

Re: Some key notes from today's Databricks Azure Roadmap Q4 event: - Faster merge and updates with Photon + Deletion vectors in Q3,- Unity Catalog - s...

Hubert-Dudek — Wed, 31 Aug 2022 15:23:12 GMT

You are awesome!

topic Re: Some key notes from today's Databricks Azure Roadmap Q4 event: - Faster merge and updates with Photon + Deletion vectors in Q3,- Unity Catalog - s... in Data Engineering

Some key notes from today's Databricks Azure Roadmap Q4 event: - Faster merge and updates with Photon + Deletion vectors in Q3,- Unity Catalog - s...

Re: Some key notes from today's Databricks Azure Roadmap Q4 event: - Faster merge and updates with Photon + Deletion vectors in Q3,- Unity Catalog - s...

Re: Some key notes from today's Databricks Azure Roadmap Q4 event: - Faster merge and updates with Photon + Deletion vectors in Q3,- Unity Catalog - s...

Re: Some key notes from today's Databricks Azure Roadmap Q4 event: - Faster merge and updates with Photon + Deletion vectors in Q3,- Unity Catalog - s...