Stay up-to-date with the latest announcements from Databricks. Learn about product updates, new features, and important news that impact your data analytics workflow.
You can run federated queries on data managed by Salesforce Data Cloud
You can use SSO authentication to connect to SQL Server
Query History and Node timeline are now available in System Tables
Query history show sql queries run by SQL warehouses. It’s powerful to track your queries (execution status, error, data management etc). Learn more
Node timeline captures node-level resource utilization data at minute granularity. Each record contains data for a given minute of time per instance. Learn more
Partition metadata logging for UC external tables
For DBR13.3 LTS or above, you can enable partition metadata logging, which is a partition discovery strategy for external tables registered to Unity Catalog. The behavior is consistent with the partition discovery strategy used in Hive metastore and only impacts Unity Catalog external tables that have partitions and use Parquet, ORC, CSV, or JSON.
Databricks recommends enabling the new behavior for improved read speeds and query performance for these tables.Learn more
What’s new in Delta Sharing ?
Delta Sharing lets you share tables where liquid clustering is enabled as well as the objects metadata including comments and primary constraints and AI Models.
LakeFlow Connect offers native connectors that enable you to ingest data from databases and enterprise applications and load it into Databricks. LakeFlow Connect leverages efficient incremental reads and writes to make data ingestion faster, scalable, and more cost-efficient, while your data remains fresh for downstream consumption.
Salesforce Sales Cloud, Microsoft Azure SQL Database, Amazon RDS for SQL Server, and Workday Reports-as-a-Service (RaaS) are currently supported.
Databricks Serverless Compute for Workflows and Notebooks are GA
Configuring and managing compute such as Spark clusters has long been a challenge for data engineers and data scientists. Time spent on configuring and managing compute is time not spent providing value to the business.
Databricks Connect for Python now supports Serverless Compute
Data Warehousing
Predictive Optimisation, which can improve your query performance by 2x through intelligent optimization of data layouts, is now in GA. Learn more
Cost management dashboards are now in Public preview, making it easy to import a dashboard to monitor costs on a workspace or account level. Learn more
In a nutshell
In Databricks Runtime 15.4 LTS and above, Scala is generally available on shared access mode Unity Catalog-enabled compute, including support for scalar user-defined functions (UDFs)
End of life for Databricks managed passwords. Learn more
Databricks provides an open source software (OSS) JDBC driver that enables you to connect tools such as DataGrip, DBeaver, and SQL Workbench/J to Databricks through Java Database Connectivity (JDBC), an industry-standard specification for accessing database management systems.
Library installation on clusters now has a timeout of 2 hours.
