Weekly Release Notes Recap
Here’s a quick recap of the latest release notes updates from the past one week.
Databricks platform release notes
April 14 - 19, 2023
Databricks Connect V2 (Public Preview)
Beginning with Databricks Runtime 13.0, Databricks Connect builds upon Spark Connect, enabling developers to use the power of Databricks and their IDE of choice to interactively develop and debug their code in any Databricks workspace.
Audit log entries for changed admin settings for workspaces and accounts
There are now audit log events for changes to some admin settings in the workspace-level logs (for workspace admin settings) and the account-level logs (for account admin settings). Within the accounts service for both types of logs, the new event action names are deleteSetting and setSetting. See Configure audit logging. Not all settings generate auditable events.
Workspaces with security profile or ESM include audit logs rows for system and monitor logs
For a workspace that uses the compliance security profile (required for some compliance programs such as FedRAMP, PCI, and HIPAA) or Enhanced Security Monitoring, there are new audit log events for the system log and monitor log. See Audit log schemas for Capsule8 and ClamAV.
Databricks SQL release notes
April 14 - 19, 2023
Databricks SQL Version 2023.20 Available
Rollout Schedule
- Preview rollout for 2023.20: Between Mar 15, 2023 and Mar 23, 2023
- Current rollout for 2023.20: Between Mar 27, 2023 and Apr 3, 2023
Changes in 2023.20
- Delta Lake schema evolution supports specifying source columns in merge statements.
- Remove all NULL elements from an array using array_compact.
- To append elements to an array, use array_append.
- To anonymize sensitive string values, use the mask function.
- Common error conditions now return SQLSTATE.
- Invoke table-valued generator functions in the regular FROM clause of a query.
- Use the from_protobuf and to_protobuf functions to exchange data between binary and struct types. See Read and write protocol buffers.
- Improved consistency for Delta commit behavior for empty transactions relating to update, delete, and merge commands.
- Behavior change
- The lateral column alias feature introduces behavior changes during name resolution. See Breaking changes.
User interface updates
The features listed in this section are independent of the SQL Warehouse compute versions described in the Channels section of the release notes.
April 18, 2023
New feature:
- Return text generated by a selected large language model (LLM) given the prompt with ai_generate_text. This function is only available as public preview on Databricks SQL Pro and Serverless. To participate in the public preview, populate and submit the AI Functions Public Preview enrollment form.
April 13, 2023
New feature:
- The TIMESTAMP_NTZ type represents values comprising of fields year, month, day, hour, minute, and second. All operations are performed regardless of time zone. See TIMESTAMP_NTZ type.
Improvements:
- Users can now send formatted results within alerts by using the QUERY_RESULT_TABLE tag in a custom alerts message.
- Users can now view the file data size for Unity Catalog tables in Data Explorer.
April 6, 2023
Improvements:
- Data explorer now supports Hive metastore delete.
- The message for the visualization editor’s initial state changed from Missing required columns to No Data.
Databricks Connect release notes
April 14 - 19, 2023
This page lists releases and maintenance updates issued for Databricks Connect.
The Databricks Connect major and minor package version must always match your Databricks Runtime version. Databricks recommends that you always use the most recent patch version of Databricks Connect that matches your Databricks Runtime version. For example, when you use a Databricks Runtime 7.3 cluster, use the latest databricks-connect==7.3.* package.
Databricks Connect for Databricks Runtime 11.3 LTS
Databricks Connect 11.3.10
April 11, 2023
- Databricks Connect client update to support Databricks Runtime 11.3 maintenance release.
