cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Missing Fields in Databricks REST API Documentation & SDK Due to OpenAPI Spec Gaps

costi9992
New Contributor III

Hi Community,

I've been working with the Databricks REST APIs and noticed some inconsistencies between the API documentation and the actual API responses. Specifically, there are a few fields returned in the API responses that are not documented but are present in the API response samples.

The missing fields are as follows:

For /api/2.1/clusters/list https://docs.databricks.com/api/workspace/clusters/list:
   - `disk_spec`
   - `last_activity_time`

For /api/2.1/clusters/get https://docs.databricks.com/api/workspace/clusters/get:
  - `disk_spec`
  - `last_activity_time`

For /api/2.1/clusters/list-node-types https://docs.databricks.com/api/workspace/clusters/listnodetypes:
  - `node_instance_type.instance_family`

For /api/2.0/pipelines https://docs.databricks.com/api/workspace/pipelines/listpipelines:
- `prev_page_token`

These fields are crucial for certain use cases, and it would be helpful to have them officially documented in the API documentation.

Moreover, this issue also impacts the **Databricks SDK**, as the SDK classes are generated from OpenAPI specs using the **Databricks SDK Generator**. Since these fields are missing from the OpenAPI spec, we cannot access them when using the SDK, which limits functionality in certain scenarios.

Has anyone else encountered this issue? Is there a plan to update the documentation and the OpenAPI specs to include these fields so they can be used in both the API and SDK?

Thanks in advance for your help!

0 REPLIES 0

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group