Performance issue while calling Sagemaker Endpoint in pyspark udf
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-30-2023 07:53 AM
Hi,
I have pyspark dataframe which calls pyspark udf which in turn calls sagemaker endpoint. But when dataframe has more rows, endpoint start failing. Also it takes longer to process.
Please suggest how to call sagemaker endpoint from pyspark.
Regards,
Sanjay