Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
DBSQL supports the following types of query caching which can help with the query performance - 1). Databricks SQL UI caching2). Query results caching3). Delta cachingPlease check here for details - https://docs.databricks.com/sql/admin/query-caching...
Are there workarounds that would allow multiple people to edit the same dashboard? I understand that "can edit" permission is on the roadmap but is there anything else they can do in the meantime to collaborate? Is it possible to disable access contr...
How would Multi Cluster Load Balancer handle JDBC requests? Does a new cluster spin up when ~10 connections are hit or is this unique to # active users?
I see that cluster sizes are mentioned here https://docs.databricks.com/sql/admin/sql-endpoints.html#cluster-size but I would like to know when to pick what type of cluster (data size/ users/ concurrency) without having to do too much trial and error...
I have two jars with the same class name. It works fine on yarn. When trying to run these jars on the Databricks cluster, I run into issues. Why Databricks is having this limitation?
When you run the jobs in Yarn, those are 2 different applications getting submitted on Yarn. Hence each application will have a separate Spark driver JVM's. In Databricks, a cluster has one JVM for the Spark driver. When applications with the same na...
There is not a default timeout to prevent long running queries.
Does Koalas support Structured Streaming
No, Koalas does not support Structured Streaming officially.As a workaround, you can use Koalas APIs with foreachBatch in Structured Streaming which allows batch APIs:>>> def func(batch_df, batch_id): ... koalas_df = ks.DataFrame(batch_df) .....
So if you've got an S3 bucket with your data in it, the first thing you'll need to do is connect it to a Databricks workspace to grant access. Then you can start querying the contents of the bucket from notebooks (or running jobs) by using clusters (...
