The error you are encountering, java.lang.NoSuchMethodError: 'org.apache.spark.sql.SparkSession$Builder org.apache.spark.sql.SparkSession$Builder.client(org.apache.spark.sql.connect.client.SparkConnectClient)', suggests that there is a mismatch between the versions of the libraries you are using.

Check Library Versions: Ensure that the versions of spark-core and databricks-connect you are using are compatible with each other. According to the context, the databricks-connect package version must match the Databricks Runtime version. For example, if you are using Databricks Runtime 12.2 LTS, you should use databricks-connect==12.2.*