topic SQL Execution API Code 400 in Data Engineering

SQL Execution API Code 400

xavier20 — Wed, 02 Aug 2023 01:32:29 GMT

I am trying to execute the following command to test API but getting response 400

import json

import os

from urllib.parse import urljoin, urlencode

import pyarrow

import requests

# NOTE set debuglevel = 1 (or higher) for http debug logging

from http.client import HTTPConnection

HTTPConnection.debuglevel = 0

HOST = "adb-8788787878787887887.2.azuredatabricks.net"

URL = "https://adb-8788787878787887887.2.azuredatabricks.net/api/2.0/sql/statements/"

WAREHOUSE = "564016fc86e262g"

AUTH_TOKEN = "jshjdhsd887d8s7d87sjhdjshdjshdsjd-2"

assert URL and WAREHOUSE and AUTH_TOKEN, "Required: HOST||URL, WAREHOUSE, and AUTH_TOKEN"

# example statement big enough to exceed 5MB data requirement

limit = None

sql_statement = "SELECT X,Y,Z FROM A LIMIT 10"

payload = json.dumps({

"statement": sql_statement,

"warehouse_id": WAREHOUSE,

"wait_timeout": "500s",

"disposition": "EXTERNAL_LINKS",

"format": "ARROW_STREAM"

})

headers = {

'Content-Type': 'application/json'

}

auth=('token', AUTH_TOKEN)

response = requests.post(URL, auth=auth, headers=headers, data=payload)

Re: SQL Execution API Code 400

daniel_sahal — Wed, 02 Aug 2023 05:03:17 GMT

@xavier20
You need to put bearer token in header, just like in this example:

https://docs.databricks.com/en/sql/admin/sql-execution-tutorial.html

Re: SQL Execution API Code 400

youssefmrini — Tue, 08 Aug 2023 16:07:32 GMT

A 400 status code response indicates that the server was unable to process the request due to a client error, e.g., incorrect syntax, invalid parametersBased on the code you provided, it appears that you are trying to execute a SQL query against your Azure Databricks workspace using the Databricks SQL REST API. However, you are encountering a 400 error when you run the script.

A 400 error typically indicates that the server could not understand the request due to bad syntax or invalid arguments. Here are a few things you can check to troubleshoot this issue:

Check that the URL variable is set correctly:
Verify that the URL variable is set to the correct endpoint for your Databricks workspace. The URL should be in the format https://<databricks-instance>/api/2.0/sql/statements/.
Verify that the WAREHOUSE and AUTH_TOKEN variables are set correctly:
Make sure that the WAREHOUSE and AUTH_TOKEN variables are set to valid values for your Databricks workspace. You can verify these values in the Databricks workspace UI or by using the Databricks CLI.
Verify the validity of the SQL statement:
Make sure that the SQL statement you are trying to execute is valid. You can test the statement directly in the Databricks workspace using the SQL editor.
Check for typos or syntax errors:
Review your code for any typos or syntax errors that may be causing the issue. Make sure that all variables and values are spelled correctly and formatted properly.

If you are still encountering issues after reviewing these areas, it may be helpful to review the error message returned from the API call to get more information on the specific issue. You can do this using the response.content property of the requests.post() call, which will display the raw response content from the API endpoint.