cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Weekly Release Notes RecapHere’s a quick recap of the latest release notes updates from the past one week. Databricks platform release notes December ...

Sujitha
Community Manager
Community Manager

Weekly Release Notes Recap

Here’s a quick recap of the latest release notes updates from the past one week.

Databricks platform release notes 

December 5 - 16, 2022

Databricks JDBC driver 2.6.32

Version 2.6.32 of the Databricks JDBC driver (download and Maven) is now available. This release updates the Jackson JSON parser libraries for enhanced security:

  • jackson-annotations-2.13.4 (previously 2.13.2)
    • jackson-core-2.13.4 (previously 2.13.2)
    • jackson-databind-2.13.4.2 (previously 2.13.2.2)

This release also resolves the following issues:

  • When using cloud fetch, the driver now cleans up certain resources properly.
    • The driver previously included cloud fetch’s presigned URLs in its log file. These presigned URLs included sensitive tokens. These log entries now exclude these tokens.
    • The driver logs a SQL statement whenever its execution hits an exception. Statements such as COPY INTO might contain secrets (for example WITH CREDENTIAL). Driver log entries now exclude such credentials.

For full details, see release-notes.txt included with the driver.

Databricks ODBC driver 2.6.29

Version 2.6.29 of the Databricks ODBC driver (download) is now available. This release adds support for Unity catalog’s primary and foreign keys, thereby enabling BI tools that use the ODBC driver to discover relationships between tables during data modeling.

This release also resolves the following issue:

  • The driver previously included cloud fetch’s presigned URLs in its log file when EnableCurlDebugLogging is set. These presigned URLs included sensitive tokens. Driver log entries now exclude these tokens.

For full details, see release-notes.txt included with the driver.

Jobs are now available in global search

You can now search for jobs in the top bar of your Databricks workspace.

Capturing lineage data with Unity Catalog is now generally available

Databricks is pleased to announce the general availability of Unity Catalog support for capturing and viewing lineage data. See Capture and view data lineage with Unity Catalog.

Databricks SQL release notes

December 8, 2022

Web application user interface updates

  • Databricks SQL alerts now support alerts for aggregations of query result columns such as SUM, COUNT, and AVG.
  • The default visualization title is now ‘VisualizationName - QueryName’ when creating new widgets on dashboards.

Databricks Connect for Databricks Runtime 11.3 LTS

December 8, 2022

Databricks Connect 11.3.0b0

  • Initial Databricks Connect client release to support Databricks Runtime 11.3 LTS.
  • There is a known issue in the R integration (some Spark plans will fail to execute).

Delta Live Tables release notes

December 6, 2022

Release 2022.49

Changes

  • You can now import files from a Databricks Repo as Python modules. You can import files from the current directory or a specified location using sys.path.append().
  • Enhanced Autoscaling is now generally available. With this release, Enhanced Autoscaling is the default autoscaling mode for new pipelines created in the UI. Databricks recommends updating pipelines that use the existing cluster autoscaling to use Enhanced Autoscaling. For more information about Enhanced Autoscaling, see Databricks Enhanced Autoscaling. For more information about configuring Enhanced Autoscaling when you create or edit a pipeline, see Create, run, and manage Delta Live Tables pipelines.
  • For pipelines run in continuous mode, the default retry limit is now 2 when retryable failures occur. Including the original attempt, all flows have three attempts (with exponential backoff) to run when non-fatal failures occur. Previously, the defaults were unbounded retries for batch flows, and five retries for streaming flows. See Delta Live Tables settings.
  • The configuration parameter pipelines.maxFlowRetryAttempts now applies to the retry limit for all flows, including batch and streaming flows, and continuous or triggered pipelines. Previously, pipelines.maxFlowRetryAttempts applied to all retry limits in triggered mode but only batch flows in continuous mode; retries for streaming pipelines in continuous mode were controlled by pipelines.numStreamRetryAttempts. For existing pipelines, pipelines.numStreamRetryAttempts continues to be supported for streaming flows in continuous mode. If both pipelines.numStreamRetryAttempts and pipelines.maxFlowRetryAttempts are set, pipelines.numStreamRetryAttempts takes priority for streaming flows in continuous mode. Otherwise, the configured value for pipelines.maxFlowRetryAttempts is used. See Delta Live Tables settings.

Bug fixes

  • This release fixes a bug causing display of incorrect DBU per hour calculations for Photon-enabled clusters.
  • This release fixes a bug causing Delta Live Tables maintenance tasks to fail in Unity Catalog enabled workspaces.
  • This release fixes a bug where stopping or restarting a continuous pipeline that ingests data with Auto Loader causes the pipeline to hang for several minutes.

Do let us know your comments below.

2 REPLIES 2

Ajay-Pandey
Esteemed Contributor III

Thanks for sharing

Harun
Honored Contributor

Thanks for sharing @Sujitha Ramamoorthy​ 

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!