Databricks Community

CarterM · 09-29-2022

I am attempting to stream JSON endpoint responses from an s3 bucket into a spark DLT. I have been very successful in this practice previously, but the difference this time is that I am storing the responses from multiple endpoints in the same s3 buck...

CarterM · 04-10-2023

@Vidula Khanna @Vidula Khanna We are experiencing the same issue in our Workflows and I was wondering if there has been any update.We need the functionality to call a method similar to `dbutils.notebook.exit` in a notebook that will cancel the exec...

CarterM · 09-30-2022

The Soccer Seasons Endpoint Response was the only one with backslashes `\` which signify a new line character. When using AutoLoader you need to specify the `multiLine` Option to indicate the JSON spans multiple lines..option("multiline", "true")This...

Databricks Community

User Stats

User Activity

Why Spark Streaming from S3 is returning thousands of files when there are only 9?

Re: How to exit the entire job in the orchestration scenario?

Re: Why Spark Streaming from S3 is returning thousands of files when there are only 9?