We figured out the issue finally! We checked the database sql audit logs and noticed that there was a particular query that was taking too long (4min) for the ingestion user. This was causing a timeout. This query is very simple and takes usually a couple seconds or even less for a DB owner: 

SELECT DISTINCT SCHEMA_NAME(schema_id) FROM sys.objects WHERE type_desc = 'USER_TABLE'
AND is_ms_shipped = 0;

 We had too many objects in our DB and the ingestion user had to go through a lot of security checks for each object. 

The solution was to clean up our DB from the unused objects and we communicated an optimized query to the databricks team to consider (which took 16sec without the cleanup):

SELECT DISTINCT s.name
FROM sys.tables t
INNER JOIN sys.schemas s ON t.schema_id = s.schema_id
WHERE t.is_ms_shipped = 0;

thank you all for your suggestions!