Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-24-2024 02:41 AM
Have you tried to pass the schema of the target table instead?
Something like:
(
spark.readStream
.format('cloudFiles')
.schema(spark.read.load(f'{target_s3_bucket}/{target_table_name}').schema)
...
)