cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks SQL Statement Execution API

RamozanbekS
New Contributor III

I'm trying to follow the example provided here https://github.com/databricks-demos/dbsql-rest-api/blob/main/python/external_links.py

It fails when it comes to downloading the data chunks. The statement status turns from SUCCEEDED to CLOSED right away without giving me time to download the data.

Here's the error message I get

<Response [404]>
b'{"error_code":"NOT_FOUND","message":"The result of the statement 01ee2ae9-da85-1e4f-a760-0abb04e34beb was not found.","details":[{"@type":"type.googleapis.com/google.rpc.ResourceInfo","resource_type":"result","resource_name":"01ee2ae9-da85-1e4f-a760-0abb04e34beb","owner":"","description":"the statement state should be SUCCEEDED before you can start fetching the result"}]}'

How can I configure my queries or my sql warehouse so that my queries have at some time to be downloaded?

1 ACCEPTED SOLUTION

Accepted Solutions

RamozanbekS
New Contributor III

It turns out that if the response is small and can fit 16mb limit, then status check will also provide single external link to download the data.

So I need a condition here. Maybe even something like this

if len(chunks) == 1:
    external_url = response.json()['result']['external_links'][0]['external_link']

 

View solution in original post

1 REPLY 1

RamozanbekS
New Contributor III

It turns out that if the response is small and can fit 16mb limit, then status check will also provide single external link to download the data.

So I need a condition here. Maybe even something like this

if len(chunks) == 1:
    external_url = response.json()['result']['external_links'][0]['external_link']

 

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!