safonov
New Contributor II

I have the same issue, but with databricks-connect==13.2.1
Code to reproduce:

col_num = 49
data = [tuple([f'val_{n}' for n in range(1, col_num + 1)])]
df = spark.createDataFrame(data=data)
for i in range(1, len(data[0]) + 1):
    df = df.withColumnRenamed(f'_{i}', f'col_{i}')
df.printSchema()

The error is a bit different though:

pyspark.errors.exceptions.connect.SparkConnectGrpcException: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNKNOWN
details = ""
debug_error_string = "UNKNOWN:Error received from peer {created_time:"2023-08-26T12:06:04.41660107+02:00", grpc_status:2, grpc_message:""}"
>

And if col_num == 48, everytning is fine. If col_num > 49, I will get a segfault without any error message.