Hi,

Quite excited to see the new release of databricks-connect, I started writing unit tests running pyspark on a databricks cluster using databricks-connect.

After some successful basic unit tests, I tested just more chained transformations on a dataframe including some forward fills, simple arithmetics, linear regressions slopes calculations via pandas udf. Nothing fancy. Then when running a test, I got the following error:

E           pyspark.errors.exceptions.connect.SparkConnectGrpcException: <_MultiThreadedRendezvous of RPC that terminated with:
E           	status = StatusCode.UNKNOWN
E           	details = ""
E           	debug_error_string = "UNKNOWN:Error received from peer  {grpc_message:"", grpc_status:2, created_time:"2023-07-06T13:29:00.033340701+00:00"}"

I do not get this error when I remove one simple column (a constant literal), and I do not get this error either if I run the same code on Databricks directly.

The error seems to point to grpc and a limitation of databricks-connect. Has anyone encountered this, and is there a place where we can check what current limitations of databricks-connect are?