Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-05-2025 01:33 AM
Hello everyone,
I am having problems working with this approach and
I am not sure if it is related to the recent changes in DLT Direct Publish (moving it to Public Preview) or if it is related to the changes I have made.
I recently bound my existing DLT to an Asset Bundle deployment and for some reason I cannot find the event_log table anymore when I run the following SQL:
SELECT *
FROM ca_catalog.gold.__<pipeline_id>_event_log
My DLT Setting are shown below as the yml configuration:
resources:
pipelines:
pipeline_dev_<user>_ca_etl_pipeline:
name: "[dev <user>] ca_etl_pipeline"
libraries:
- notebook:
path: /Workspace/Shared/pipeline/src/jobs/dlt_pipeline/generic_dlt_pipeline
schema: gold
development: true
photon: true
channel: PREVIEW
catalog: ca_catalog
serverless: true
deployment:
kind: BUNDLE
metadata_file_path: /Workspace/Users/<user>/.bundle/ca_etl_pipeline_databricks_bundle/dev/state/metadata.json
resources:
pipelines:
pipeline_dev_<user>_ca_etl_pipeline:
name: "[dev <user>] ca_etl_pipeline"
libraries:
- notebook:
path: /Workspace/Shared/pipeline/src/jobs/dlt_pipeline/generic_dlt_pipeline
schema: gold
development: true
photon: true
channel: PREVIEW
catalog: ca_catalog
serverless: true
deployment:
kind: BUNDLE
metadata_file_path: /Workspace/Users/<user>/.bundle/ca_etl_pipeline_databricks_bundle/dev/state/metadata.json
Could it have something to do with my asset bundle deployment or is it a known problem in Databricks?
