Databricks

CarterM · 09-29-2022

I am attempting to stream JSON endpoint responses from an s3 bucket into a spark DLT. I have been very successful in this practice previously, but the difference this time is that I am storing the responses from multiple endpoints in the same s3 buck...

CarterM · 04-10-2023

@Vidula Khanna @Vidula Khanna We are experiencing the same issue in our Workflows and I was wondering if there has been any update.We need the functionality to call a method similar to `dbutils.notebook.exit` in a notebook that will cancel the exec...

CarterM · 09-30-2022

The Soccer Seasons Endpoint Response was the only one with backslashes `\` which signify a new line character. When using AutoLoader you need to specify the `multiLine` Option to indicate the JSON spans multiple lines..option("multiline", "true")This...

Databricks

User Stats

User Activity

Why Spark Streaming from S3 is returning thousands of files when there are only 9?

Re: How to exit the entire job in the orchestration scenario?

Re: Why Spark Streaming from S3 is returning thousands of files when there are only 9?