Databricks Rest API Statement Execution - External Links

RobsonNLPT
Contributor III

Hi.

I've tested the adb Rest Api to execute queries on databricks sql serverless. Using INLINE as disposition I have the json array with my correct results but using EXTERNAL_LINKS I have the chunks but the external_link (URL starting with http://storage-proxy.databricks.com) doesn't work (This site can’t be reached)

Any help?

 

RobsonNLPT
Contributor III

By the way I can acess all chunks using the endpoint /api/2.0/sql/statements/{statement_id}/result/chunks/{chunk_index}

mark_ott
Databricks Employee
Databricks Employee

The issue you're experiencing with Databricks SQL Serverless REST API in EXTERNAL_LINKS mode—where the external_link URL (http://storage-proxy.databricks.com/...) does not work, but you can access chunks directly via the /api/2.0/sql/statements/{statement_id}/result/chunks/{chunk_index} endpoint—is reported by other users as well. This external_link is supposed to let you download result files directly, but in some environments, especially with strict firewall/proxy rules or misconfigured network access, these links can be inaccessible.​

Key Insights

  • The external_link URLs usually point to Databricks' storage-proxy, which must be reachable from your network. If it is not, often due to proxy/firewall restrictions, you get the "site can’t be reached" error.​

  • You are correct in using the chunk API directly (/api/2.0/sql/statements/{statement_id}/result/chunks/{chunk_index}) as a workaround. Databricks documentation supports this approach, especially for environments where external_link URLs are not accessible.​

  • When using EXTERNAL_LINKS disposition, Databricks may not always return a next_chunk_index as expected, which seems to be a quirk or a bug in some cases. However, as you've found, iterating over the manifest's chunk indexes manually still works.​

  • You do not need to use the external_link if the internal chunk API endpoints are working; these should be reliable as long as you have appropriate API authentication and permissions.​

Recommendations

  • Continue using /api/2.0/sql/statements/{statement_id}/result/chunks/{chunk_index} to reliably fetch your results, especially if the external_link is not reachable from your network.​

  • If you need to make external_link URLs work, check with your network/security admin to ensure traffic to *.databricks.com and specifically storage-proxy.databricks.com is allowlisted, and confirm no corporate proxies or firewalls are blocking access.​

  • Monitor the Databricks release notes and known issues for updates, as this is a recurring concern and may be addressed in future platform revisions.

Your workaround is valid and in line with current documented and community-recommended practices for working around the external_link accessibility issue