cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
patojo94
New Contributor II
since ‎05-04-2022
‎11-28-2023

User Stats

  • 3 Posts
  • 0 Solutions
  • 0 Kudos given
  • 3 Kudos received

User Activity

Stream failure JsonParseException

11-28-2023
Hi all! I am having the following issue with a couple of pyspark streams. I have some notebooks running each of them an independent file structured streaming using  delta bronze table  (gzip parquet files) dumped from kinesis to S3 in a previous job....

pyspark streaming failed for now reason

05-07-2022
Hi everyone, I have a pyspark streaming reading from an aws kinesis that suddenly failed for no reason (I mean, we did not make any changes in the last time).It is giving the following error: ERROR MicroBatchExecution: Query kinesis_events_prod_bronz...

Adding deduplication method to spark streaming

05-04-2022
Hi everyone, I am having some troubles to add a deduplication step on a file streaming that is already running. The code I am trying to add is this one:df = df.withWatermark("arrival_time", "20 minutes")\ .dropDuplicates(["event_id", "arrival_time"])...
Kudos from