Databricks

Hi Databricks community team,I have code as below"""df = spark.readStream \.format("kinesis") \.option("endpointUrl", endpoint_url) \.option("streamName", stream_name) \.option("initialPosition", "latest") \.option("consumerMode", "efo") \.option("ma...

Hi Databricks community,I'm using Databricks Jobs Cluster to run some jobs. I'm setting the worker and driver type to AWS m6gd.large, which has 2 cores and 8G of memory each.After seeing it's defaulting executor memory to 2G, I wanted to increase it,...