Performance issue while calling Sagemaker Endpoint in pyspark udf

sanjay
Valued Contributor II

Hi,

I have pyspark dataframe which calls pyspark udf which in turn calls sagemaker endpoint. But when dataframe has more rows, endpoint start failing. Also it takes longer to process.

Please suggest how to call sagemaker endpoint from pyspark.

Regards,

Sanjay