I see. So maybe the session isolation is causing issues with library compatibility or networking, still interesting to understand why for newer library versions only. It might be due to classloading restrictions in isolated sessions preventing the connector from loading necessary dependencies properly; or networking configurations for Spark Connect which might introduce connection issues, such as the "connection refused."

If you'd like to further investigate this, I'd suggest raising a support ticket for better close collaboration on root causing this behavior.

Thanks for your feedback!