I have a delta live table pipeline which works fine on normal DLT job cluster.

But as soon as we switch it to use serverless compute it fails.

The failure happens at "import pymssql" after doing pip install pymssql as first statement of the source code notebook of the DLT Pipeline.

The error message is not helpful at all, but i tried to narrow it down to this statement after number of executions.

Is it a bug in DLT Serverless runtime? the same code works on serverless spark cluster as well