cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Feature store feature table location

Direo
Contributor

Can Databricks feature tables be stored outside of DBFS?

3 REPLIES 3

Priyag1
Honored Contributor II

@Direo Direo​ Feature tables are delta tables it can shared among different workspaces . But you are asking outside of dbfs , what exactly is requirement and where you want to store

Anonymous
Not applicable

Hi @Direo Direo​ 

Does @Priyadarshini G​  answer help? If it does, would you be happy to mark it as best? If it doesn't, please tell us so we can help you.

the-sab
New Contributor II

Yes, Databricks feature tables can be stored outside of Databricks File System (DBFS). You can store your feature tables in external storage systems such as Amazon S3, Azure Blob Storage, Azure Data Lake Storage, or Hadoop Distributed File System (HDFS).

To store your feature tables in external storage, you need to configure the storage system and provide the appropriate connection information when creating your Delta table. For example, when using Amazon S3, you would specify the S3 bucket path when creating the table.

Here's an example of how to create a Delta table in an Amazon S3 bucket using PySpark:

```python
from pyspark.sql import SparkSession

# Start a Spark session
spark = SparkSession.builder \
.appName("Databricks Feature Table on S3") \
.getOrCreate()

# Define a sample DataFrame
data = [("Alice", 34), ("Bob", 45), ("Cathy", 29)]
columns = ["Name", "Age"]
df = spark.createDataFrame(data, columns)

# Write the DataFrame to a Delta table in S3
delta_table_path = "s3a://your-bucket-name/your-delta-table-path/"
df.write.format("delta").mode("overwrite").save(delta_table_path)
```

Replace `your-bucket-name` and `your-delta-table-path` with the appropriate values for your Amazon S3 bucket and desired path. Note that you need to configure your S3 authentication and ensure that you have the necessary permissions to read and write to the specified bucket.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!