Hi @QueryingQuail,

The behavior you are seeing is expected. Lakeflow Spark Declarative Pipelines (SDP), previously known as DLT, do not currently support reading directly from Lakehouse Federation foreign catalogs. The DATA_SOURCE_NOT_FOUND / UNKNOWN_CONNECTION_TYPE error occurs because the SDP pipeline runtime does not have the federation connector needed to resolve the foreign catalog connection at query time.

For reference, Unity Catalog-enabled SDP pipelines can read from these data sources:

- Unity Catalog managed and external tables, views, materialized views, and streaming tables
- Hive metastore tables and views
- Auto Loader (using the read_files() function) from Unity Catalog external locations
- Apache Kafka and Amazon Kinesis

Foreign catalogs (including OneLake connections) are not in that supported list. This applies regardless of the foreign catalog type.

You can find the supported sources documented here:
https://docs.databricks.com/en/delta-live-tables/unity-catalog.html

The "https://internal/" URL you noticed in the connection details is normal for OneLake catalog federation connections, so that part is not the issue.

WORKAROUND

The recommended approach is to stage the data from your foreign catalog into a regular Unity Catalog table first, and then have your SDP pipeline read from that UC table. You can do this with a scheduled Lakeflow Job (or a simple notebook) that runs a query like:

CREATE OR REPLACE TABLE my_catalog.my_schema.onelake_buildingcase
AS SELECT * FROM fabric.dbo.nnit_buildingcase

Then update your pipeline code to read from the staged table:

import dlt

@dlt.table
def fabric_test():
  return spark.read.table("my_catalog.my_schema.onelake_buildingcase")

If you need incremental/streaming behavior, you could use a MERGE statement in the staging job to handle updates, or use Auto Loader if the source data is available as files in cloud storage.

ALTERNATIVE: NOTEBOOK TASK IN THE SAME JOB

If you want to keep everything in a single workflow, you can create a Lakeflow Job with two tasks:

1. A notebook task that queries the foreign catalog and writes to a UC table
2. Your SDP pipeline task (dependent on task 1) that reads from that UC table

This gives you a single orchestrated workflow while working within the current supported data source boundaries for SDP pipelines.

* This reply used an agent system I built to research and draft this response based on the wide set of documentation I have available and previous memory. I personally review the draft for any obvious issues and for monitoring system reliability and update it when I detect any drift, but there is still a small chance that something is inaccurate, especially if you are experimenting with brand new features.

If this answer resolves your question, could you mark it as "Accept as Solution"? That helps other users quickly find the correct fix.