LeoGallerDbx
Databricks Employee
Databricks Employee

Thanks for the extra details – this is very helpful.

From what you shared, your service principal already has the right permissions inside the target database (SELECT on your tables, VIEW CHANGE TRACKING, VIEW DEFINITION, VIEW DATABASE STATE, sys.* grants, etc.), and the setup script ran successfully. That matches what we expect for the ingestion user.

For Azure SQL Database, there is one additional requirement that isn’t obvious: the ingestion login must have access to the master database so Lakeflow can discover databases and run some metadata queries. The recommended way to do this is to add the login to the built‑in ##MS_DatabaseConnector## role in master:

-- Run in the master database
ALTER ROLE ##MS_DatabaseConnector## 
ADD MEMBER [service-principle-login];
 

Alternatively (if you can’t use that role), you can:

  • Create a login in master (if not already present), and
  • Create a user for that login in `master` with minimal permissions, and then a user in the target database with the documented grants.

    The full list of required permissions is documented here under “Azure SQL Database”:

    After you add the master‑db access (via ##MS_DatabaseConnector## or the alternative), please:

  1. Restart the ingestion gateway pipeline.
  2. Re‑run the **Database setup → Validate** step.

    If it still times out, the next step would be to grab the gateway driver logs (stdout / log4j) from the ingestion gateway job and look for permission or “System access validation” messages – feel free to paste those here, and we can dig deeper.

Let us know if it works.

Leonardo Galler