Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-13-2022 10:24 PM
You can write your ETL logic in notebooks, run the notebook over a cluster and write the data to a location where your S3 bucket is mounted.
Next, you can register that table with Hive MetaStore and access the same table in Databricks SQL.
To see the table, go to Data tab and select your schema/database to see registered tables.
Two ways to do this:
Option 1:
df.write.option("path",<s3-path-of-table>).saveAsTable(tableName)Option 2
%python
df.write.save(<s3-path-of-table>)
%sql
CREATE TABLE <table-name>
USING DELTA
LOCATION <s3-path-of-table>: