Delta Live Tables - RETRY_ON_FAILURE

GuMart — Thu, 13 Apr 2023 07:31:12 GMT

Hi,

Is it possible to set it up the RETRY_ON_FAILURE property for DLTs through the API?

I'm not finding in the Docs (although it seems to exist in a response payload).

https://docs.databricks.com/delta-live-tables/api-guide.html

Re: Delta Live Tables - RETRY_ON_FAILURE

Anonymous — Sun, 16 Apr 2023 07:11:18 GMT

@Gustavo Martins :

Yes, you can set the RETRY_ON_FAILURE property for a Delta Live Table (DLT) using the API. You can set the retry_on_failure parameter when creating or updating a DLT using the dlt.create or dlt.update API calls respectively. Here's an example of how you can set the retry_on_failure property to true:

import requests
import json
 
api_token = '<YOUR_API_TOKEN>'
endpoint = 'https://<YOUR_DATABRICKS_INSTANCE>/api/2.0/preview/dlt'
 
table_name = 'my_table'
retry_on_failure = True
 
# Set up the request data
request_data = {
    'table_name': table_name,
    'retry_on_failure': retry_on_failure
}
 
# Set up the request headers
headers = {
    'Authorization': f'Bearer {api_token}',
    'Content-Type': 'application/json'
}
 
# Create the DLT
response = requests.post(endpoint + '/create', headers=headers, data=json.dumps(request_data))
 
# Check the response
if response.ok:
    print('DLT created successfully')
else:
    print('DLT creation failed with status code', response.status_code, response.text)

Similarly, you can update the retry_on_failure property of an existing DLT using the dlt.update API call.

Here's an example of how you can update the retry_on_failure property of an existing DLT:

import requests
import json
 
api_token = '<YOUR_API_TOKEN>'
endpoint = 'https://<YOUR_DATABRICKS_INSTANCE>/api/2.0/preview/dlt'
 
table_name = 'my_table'
retry_on_failure = True
 
# Set up the request data
request_data = {
    'table_name': table_name,
    'retry_on_failure': retry_on_failure
}
 
# Set up the request headers
headers = {
    'Authorization': f'Bearer {api_token}',
    'Content-Type': 'application/json'
}
 
# Update the DLT
response = requests.post(endpoint + '/update', headers=headers, data=json.dumps(request_data))
 
# Check the response
if response.ok:
    print('DLT updated successfully')
else:
    print('DLT update failed with status code', response.status_code, response.text)

Re: Delta Live Tables - RETRY_ON_FAILURE

GuMart — Mon, 17 Apr 2023 05:58:19 GMT

Hi @Suteja Kanuri ,

Thank you so much for the quick and complete answer!

Regards,

topic Re: Delta Live Tables - RETRY_ON_FAILURE in Data Engineering

Delta Live Tables - RETRY_ON_FAILURE

Re: Delta Live Tables - RETRY_ON_FAILURE

Re: Delta Live Tables - RETRY_ON_FAILURE