How to set pipelines.incompatibleViewCheck.enabled = false
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-16-2022 12:36 PM
I tried to load a static table as source to a streaming dlt pipeline. I understand this is not optimum, but it provides the best path toward eventually having a full streaming pipeline. When I do I get the following error:
pyspark.sql.utils.AnalysisException: View 'raw_public_streamme' is not a streaming view and must be referenced using read. This check can be disabled by setting Spark conf pipelines.incompatibleViewCheck.enabled = false.
How can I set pipelines.incompatibleViewCheck.enabled = false? Doing so with spark.conf.set does not seem to have any effect.
Thanks.
Labels:
- Labels:
-
DLT Pipeline