cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Retry api request if fails

Braxx
Contributor II

I have a simple API request to query a table and retrive data, which are then suited into a dataframe. May happened, it fails due to different reasons. How to retry it for let's say 5 times when any kind of error takes place?

Here is an api request:

df  = spark.createDataFrame(sf.bulk.MyTable.query("select Id,  Status from MyTable "))

1 REPLY 1

Manoj
Contributor II

@Bartosz Wachocki​ ,

Use timeout, retry interval ,recursion and exception handling

pseudo code below

timeout = 300

def exec_query(query,timeout):

try:

df = spark.createDataFrame(sf.bulk.MyTable.query(query))

except:

if timeout > 0 :

sleep(60)

exec_query(query)

timeout = timeout - 60

else:

print("Timeout")

break;

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.