Job aborted due to stage failure: Task 6506 in stage 46.0 failed 4 times, most recent failure: Lost task 6506.3 in stage 46.0 (TID 12896) (10.**.***.*** executor 12): java.lang.OutOfMemoryError: Cannot reserve 4194304 bytes of direct buffer memory (allocated: 5062249863, limit: 5065146368)

I am facing this issue when i run my code in databricks notebook on serverless compute. the code is reading data from table (700 million) and ingesting rows to api in batches, after getting response from api, failed batched i am storing into other table, after ingestion 250 million records i am getiing this error.