topic Error when creating a delta live table pipeline in Data Engineering

Error when creating a delta live table pipeline

NM447101 — Tue, 11 Apr 2023 09:45:13 GMT

INVALID_PARAMETER_VALUE: Validation failed for node_type_id, the value must be Standard_DS3_v2 (is "Standard_F8s")

Re: Error when creating a delta live table pipeline

NM447101 — Tue, 11 Apr 2023 11:05:00 GMT

specify node_type_id in the json file when creating the pipeline

Re: Error when creating a delta live table pipeline

Tayyab_Vohra — Tue, 11 Apr 2023 11:12:05 GMT

The error message indicates that there is an issue with the node type that you have specified for your Delta Live Table pipeline in Databricks. Specifically, the node type must be set to "Standard_DS3_v2", but the error message indicates that it has been set to "Standard_F8s", which is not a valid node type for the pipeline.

To resolve this issue, you should update the configuration of your Delta Live Table pipeline to use a valid node type. You can typically do this by modifying the configuration file for the pipeline or updating the parameters that are passed to the pipeline when it is created.

In this case, you should update the node type to "Standard_DS3_v2". This can be done in the Databricks UI by selecting the pipeline and then updating the "node_type_id" parameter to the correct value. Alternatively, you can update the configuration file for the pipeline and then recreate the pipeline using the updated configuration.

Once you have updated the node type to a valid value, you should be able to run your Delta Live Table pipeline without encountering any validation errors.

Re: Error when creating a delta live table pipeline

Anonymous — Wed, 12 Apr 2023 06:58:07 GMT

Hi @Nitya Mehta

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!