Solution for executing Bigquery DDL in the notebook cell

[Loinguyen318]

Hi all,

I would like to find a solution to execute a Bigquery DDL in a cell of databricks notebook directly.

Have any solution for my problem?

This is the example of DDL:

CREATE EXTERNAL TABLE `PROJECT_ID.DATASET.DELTALAKE_TABLE_NAME`
WITH CONNECTION `PROJECT_ID.REGION.CONNECTION_ID`
OPTIONS (
  format ="DELTA_LAKE",
  uris=['DELTA_TABLE_GCS_BASE_PATH']);

[SP_6721]

Hi @Loinguyen318 ,

To execute BigQuery DDL from Databricks, you must use Python code in your notebook with the google-cloud-bigquery library.

from google.cloud import bigquery

client = bigquery.Client()
ddl = """
CREATE EXTERNAL TABLE `PROJECT_ID.DATASET.DELTALAKE_TABLE_NAME`
WITH CONNECTION `PROJECT_ID.REGION.CONNECTION_ID`
OPTIONS (
  format ="DELTA_LAKE",
  uris=['DELTA_TABLE_GCS_BASE_PATH']
);
"""
job = client.query(ddl)
job.result()

[SmithPoll]

You can use the google.cloud.bigquery client within a Databricks notebook by installing the google-cloud-bigquery library and using Python to run the DDL. Example:

from google.cloud import bigquery client = bigquery.Client() query = \"\"\"CREATE EXTERNAL TABLE `PROJECT_ID.DATASET.DELTALAKE_TABLE_NAME` WITH CONNECTION `PROJECT_ID.REGION.CONNECTION_ID` OPTIONS ( format ="DELTA_LAKE", uris=['DELTA_TABLE_GCS_BASE_PATH']);\"\"\" client.query(query).result()

Make sure your service account is properly configured with access.