If possible, how can one go about installing a Python library with SDK dependencies like pyRFC? (https://github.com/SAP/PyRFC)
The SDK dependencies depend on the type of OS, and since we're running Databricks out of AWS, I assume one would have to match the SDK to correspond with a Linux OS.
And how would we point the cluster to look for the SDK files needed for library compilation? Do we have to keep the SDK files in an S3 storage (outside of Databricks), or can we keep these files within DBFS?
I am assuming environment variables pointing to the location of necessary files for the library to run can be declared within the cluster creation UI.
Sidenote: Team is working on building the infra to stage our Databricks, so hopefully I can get some test results in once that is done to beef up this question, and maybe even make it a post at some point.
Would appreciate insight! Thanks!