Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
Are there workarounds that would allow multiple people to edit the same dashboard? I understand that "can edit" permission is on the roadmap but is there anything else they can do in the meantime to collaborate? Is it possible to disable access contr...
How would Multi Cluster Load Balancer handle JDBC requests? Does a new cluster spin up when ~10 connections are hit or is this unique to # active users?
I see that cluster sizes are mentioned here https://docs.databricks.com/sql/admin/sql-endpoints.html#cluster-size but I would like to know when to pick what type of cluster (data size/ users/ concurrency) without having to do too much trial and error...
I have two jars with the same class name. It works fine on yarn. When trying to run these jars on the Databricks cluster, I run into issues. Why Databricks is having this limitation?
When you run the jobs in Yarn, those are 2 different applications getting submitted on Yarn. Hence each application will have a separate Spark driver JVM's. In Databricks, a cluster has one JVM for the Spark driver. When applications with the same na...
There is not a default timeout to prevent long running queries.
Does Koalas support Structured Streaming
No, Koalas does not support Structured Streaming officially.As a workaround, you can use Koalas APIs with foreachBatch in Structured Streaming which allows batch APIs:>>> def func(batch_df, batch_id): ... koalas_df = ks.DataFrame(batch_df) .....
So if you've got an S3 bucket with your data in it, the first thing you'll need to do is connect it to a Databricks workspace to grant access. Then you can start querying the contents of the bucket from notebooks (or running jobs) by using clusters (...
Frequency of logs in stdout / stderr would be a function of what code you run on the databricks clusters .The default log level for log4j is INFO - you could change it following the instructions here
