How to load xlsx Files to Delta Live Tables (DLT)?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-13-2024 06:53 AM - edited 06-13-2024 06:54 AM
I want to load a .xlsx file to DLT but struggling as it is not available with Autoloader.
With the Assistant I tried to load the .xlsx first to a data frame and then send it to DLT.
import dlt
from pyspark.sql import SparkSession
# Load xlsx file into DataFrame
df = spark.read \
.format("com.crealytics.spark.excel") \
.option("dataAddress", "${my_etl.sheet_address}") \
.option("header", "${my_etl.header}") \
.option("inferSchema", "${my_etl.infer_schema}") \
.option("timestampFormat", "${my_etl.timestamp_format}") \
.load("${my_etl.input_path}")
@dlt.table
def conformed():
return df
but it is not working.