Imagine all a data engineer or analyst needs to do to read from a REST API is use spark.read(), no direct request calls, no manual JSON parsing - just spark .read. Thatโs the power of a custom Spark Data Source. Soon, we will see a surge of open-source connectors.
My blog: https://databrickster.medium.com/
