we do have a custom driver to pull the data out but we would need to change it for writing it in Databricks. Will try HTTP API first and if unsuccessful go with driver approach. Thanks for the nice suggestion.