Databricks Community is an open-source platform for data enthusiasts and professionals to discuss, share insights, and collaborate on everything related to Databricks. Members can ask questions, share knowledge, and support each other in an environment that ensures respectful interactions.
Hey so Ive just started to use DAB to automatically mange job configs via CICD I had a previously-existing job (lets say ID 123) which was created manually and had this config resou...
We have 3 environments (dev, qc and prod). In DEV and QC I can issue this command: SHOW GROUPS from a sql notebook and it will show the group I have created for each workspace. However in pro...
I would like to know how to connect to Azure Blob Storage in a Python job inside a workflow with serverless cluster. When working with a non-serverless cluster or with serverless in a declarative pip...
I ran the command ANALYZE TABLE <TABLE_NAME> COMPUTE STATISTICS FOR COLUMNS 'COL1,COL2' to compute statistics for specific columns, but I noticed that the distinct counts returned by this comma...
Hello, i am trying to run a databricks dashboard via workflow. when i deploy the dashboard .json file in prod workspace via import dashboard option the dashboard_id is changed. is there a way i can...
hello, im trying to get the previous version of a delta table using timestamp but databricks sql does not allow to use variables the only thing i can do is use TIMESTAMP AS OF CURRENT_DATE()...
I’m planning to take the Databricks Data Engineer Associate certification exam, and I wanted to ask if there are any official discounts, coupons, or student offers available to help reduce the exam f...
We are trying to execute notebook recently migrated to unity catalog through Pool cluster from synapse pipeline. But it giving error as "UC not enabled".Could you please suggest on this.
Hello, For one of my clients, we are using an all-purpose cluster to run some Databricks notebooks. We noticed in the logs of the cluster some Azure Quota Exceptions, from which we would like to kno...
Why can't we just copy all the DLT tables and materialized views from one UC catalog to another to get the historical data in place and then run the DLT pipelines on those UC tables? We are migratin...