cancel
Showing results for 
Search instead for 
Did you mean: 
Mathias_Peters
Contributor
since ‎01-12-2024
Sunday

User Stats

  • 12 Posts
  • 1 Solutions
  • 0 Kudos given
  • 3 Kudos received

User Activity

Hi, I implemented a job that should incrementally read all the available data from a Kinesis Data Stream and terminate afterwards. I schedule the job daily. The data retention period of the data stream is 7 days, i.e., there should be enough time to ...
Hi, I have a job consisting of three tasks:  tasks: - task_key: Kinesis_to_S3_new spark_python_task: python_file: ../src/kinesis.py parameters: ["${var.stream_region}", "${var.s3_base_path}"] j...
Hi, I am a bit stumped atm bc I cannot figure out how to get a DLT table definition picked up in a Python notebook. 1. I created a new notebook in python2. added the following code:  %python import dlt from pyspark.sql.functions import * @dlt.table(...
Hi, I am loading data from a kinesis data stream using DLT. CREATE STREAMING TABLE Consumers_kinesis_2 ( ..., unbase64(data) String, ... ) AS SELECT * FROM STREAM read_kinesis (...) Is it possible to directly cast, unbase64, and/or transform the resu...
Kudos from