- 1753 Views
- 3 replies
- 2 kudos
Resolved! log error to cosmos db
Objective:- Retrieve objects from an S3 bucket using a 'get' api call, write the retrieved object to azure datalake and in case of errors like 404s (object not found) write the error message to cosmos DB"my_dataframe" consists of the a column (s3Obje...
- 1753 Views
- 3 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi @Sandesh Puligundla​ issue is that you are using spark context inside foreachpartition. You can create a dataframe only on the spark driver. Few stack overflow references https://stackoverflow.com/questions/46964250/nullpointerexception-creatin...
- 2 kudos