python library in databricks

[chari]

Hello community members,

I am seeking to understand where databricks keeps all the python libraries ? 

For a start, I tried two lines below:

import sys 

sys.path()

This list all the paths but I cant look inside them. How is DBFS different from these paths from sys.path() ?

Regards,

Veeru

[Wojciech_BUK]

Hello,
all your libraries are installed on Databricks Cluster Driver node on OS Disk.
DBFS is like mounted Cloude Storage account.

You have veriouse ways of working with libraries but databricks only load some of libraries that comes with Cluster image.

If you want custom librariesm, you can put them e.g. to DBFS and when you launch cluster, you can install those libraries on your cluster from DBFS (or use init script).

If you install libraries from inside of notebook, this installation will persist as long as your cluster won't be terminated. 
Next time cluster starts, you need to install those libraries again as it is brand new compute with brand new disk.