Databricks Community

The following is my scenario:I need to query on a daily basis from an external table that maintains a row versionI would like to be able to query for all records where the row version is greater than the max previously processed row version. The sour...

Here I have a table named organizations_silver that was build from a bronze table created from a Kinesis change data capture feed.@dlt.table(name="kinesis_raw_stream", table_properties={"pipelines.reset.allowed": "false"})def kinesis_raw_stream(): ...

I have this behavior that I'm not understanding. I have a notebook that defines a DLT from a Kinesis stream and a view from that DLT. This works when I run it from within workflow, configured using the DLT pipeline. If, however, I create a workflow a...

I'm trying to create a pipeline containing a view from a federated source. In this case, I'd like to just create materialized views from the federation and and schedule the pipeline for execution. If I define a pipeline  with only something like the ...