Hi Suteja,

Thanks for the lengthy reply. We also think it's some kind of race condition.

As to your points...

We're not sure how to influence 1/2 since they are part of the Databricks environment.

Also, we're not convinced how 4/5/6 will help since the problem is that the request goes to the wrong server.

That leaves 1. We found there is an alternative driver and we are trying this to see if it helps.

In code:

df = (spark.read
    .format('sqlserver')
    .option('host', ip)
    .option('user', connection.user)
    .option('password', pw)
    .option('database', connection.database)
    .option('sslProtocol', 'TLSv1.2')
    .option('trustServerCertificate', 'true')
    .option('encrypt', 'true')
    .option(query_or_table, statement)
)

We need to wait to see if this resolves the problem.

Thanks,

Michel