Hi @Loinguyen318 ,To execute BigQuery DDL from Databricks, you must use Python code in your notebook with the google-cloud-bigquery library.from google.cloud import bigqueryclient = bigquery.Client()ddl = """CREATE EXTERNAL TABLE `PROJECT_ID.DATASET....
Hi @lizou1 ,Databricks serverless compute jobs automatically scale based on workload, so there’s no need to manually configure clusters, Databricks manages that for you.However, autoscaling doesn’t always resolve issues caused by data skew or shared ...
Hi @Sainath368 ,From what I understand, COMPUTE DELTA STATISTICS generates Delta statistics that are mainly used for data skipping, helping speed up table scans by avoiding unnecessary file reads. However, these stats aren't used by the query optimiz...
Hi @seefoods ,We usually use Trigger.AvailableNow when files arrive in batches rather than continuously. If your script keeps running even after processing, it could be that the job is still checking for any remaining files.If it seems to be hanging ...
Hi @AdamIH123 ,The explode-based approach is widely used and remains the most reliable and readable method.But if you're looking for an alternative without using explode, you can try the REDUCE + MAP_FILTER approach. It lets you aggregate maps across...
