- 2554 Views
- 1 replies
- 0 kudos
How many users can the JDBC endpoint support in the All Purpose HC?
What is the max number of users can the JDBC endpoint support in the All Purpose high concurrency cluster? To support more sql workloads, is it better to go with Databricks Sql Enpoints?
- 2554 Views
- 1 replies
- 0 kudos
- 0 kudos
There is a limit an execution context limit of 145. This means you can have at most 145 notebooks attached to a cluster. https://kb.databricks.com/execution/maximum-execution-context.htmlIf you are primarily using SQL then Databricks SQL Endpoints wo...
- 0 kudos
- 2586 Views
- 1 replies
- 0 kudos
Resolved! Does size of optimized files after running OPTIMIZE varies between cloud providers (S3, Blob and GCS)?
are there any other parameters to consider running OPTIMIZE depending cloud vendor?
- 2586 Views
- 1 replies
- 0 kudos
- 0 kudos
The optimize is not dependent on the cloud provider whatsoever. Optimize will produce the same results regardless of the underlying storage. It is idempotent, meaning if it is run twice on the same dataset the the second execution has no effect.
- 0 kudos
- 3993 Views
- 1 replies
- 0 kudos
- 3993 Views
- 1 replies
- 0 kudos
- 0 kudos
If you expect a column to be commonly used in query predicates and if that column has high cardinality (that is, a large number of distinct values), then use ZORDER BY.You can specify multiple columns for ZORDER BY as a comma-separated list. However,...
- 0 kudos
- 2290 Views
- 1 replies
- 0 kudos
- 2290 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Yes we have the best practices,I can point on the data bricks Docshttps://docs.databricks.com/administration-guide/access-control/tokens.html
- 0 kudos
- 2480 Views
- 1 replies
- 0 kudos
- 2480 Views
- 1 replies
- 0 kudos
- 0 kudos
we have a limit of 100 Secret scopes and each secret can have 1000 key-value pair of secrets.https://docs.microsoft.com/en-us/azure/databricks/security/secrets/secret-scopes
- 0 kudos
- 1852 Views
- 1 replies
- 0 kudos
- 1852 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi we have best practices around streaming with Kinesis and you can find them here .https://docs.databricks.com/spark/latest/structured-streaming/kinesis-best-practices.html
- 0 kudos
- 2062 Views
- 1 replies
- 0 kudos
- 2062 Views
- 1 replies
- 0 kudos
- 0 kudos
nope not possible you can set the name that shows up in azure portal though (but only when you’re first creating the WS i believe)
- 0 kudos
- 1848 Views
- 1 replies
- 0 kudos
- 1848 Views
- 1 replies
- 0 kudos
- 0 kudos
Running Jobs configured to use an Existing All-Purpose Cluster is considered to be a Job client. If you set workload_type.clients.jobs = false on an all-purpose cluster, it can not be used to run jobs.
- 0 kudos
- 12063 Views
- 1 replies
- 1 kudos
- 12063 Views
- 1 replies
- 1 kudos
- 1 kudos
In Databricks, each cluster creates an initial spark session. And each notebook creates a spark subsession within the same. A temporary View created in one notebook isn't accessible to others. If you need to share view across notebooks, you use Globa...
- 1 kudos
- 5903 Views
- 1 replies
- 0 kudos
Resolved! What is the difference between coalesce and repartition when it comes to shuffle partitions in spark
- 5903 Views
- 1 replies
- 0 kudos
- 0 kudos
Coalesce essentially groups multiple partitions into a larger partitions. So use coalesce when you want to reduce the number of partitions (and also tasks) without impacting sort order. Ex:- when you want to write-out a single CSV file output instea...
- 0 kudos
- 3275 Views
- 1 replies
- 0 kudos
- 3275 Views
- 1 replies
- 0 kudos
- 0 kudos
You can disable magic commands for a workspace but not on a per-user basis. Additional details about Workspace ACLs can be found here - https://docs.databricks.com/security/access-control/workspace-acl.html
- 0 kudos
- 10325 Views
- 1 replies
- 0 kudos
- 10325 Views
- 1 replies
- 0 kudos
- 0 kudos
Narrow Transformation: In Narrow transformation, all the elements that are required to compute the records in single partition live in the single partition of parent RDD. Ex:- Select, Filter, Union, Wide Transformation: Wide transformation, all the e...
- 0 kudos
- 5464 Views
- 1 replies
- 0 kudos
- 5464 Views
- 1 replies
- 0 kudos
- 0 kudos
Spark's execution engine is designed to be Lazy. In effect, you're first up build up your analytics/data processing request through a series of Transformations which are then executed by an ActionTransformations are kind of operations which will tran...
- 0 kudos
- 23598 Views
- 2 replies
- 0 kudos
- 23598 Views
- 2 replies
- 0 kudos
- 0 kudos
%run is copying code from another notebook and executing it within the one its called from. All variables defined in the notebook being called are therefore visible to the caller notebook dbutils.notebook.run() is more around executing different note...
- 0 kudos
- 92626 Views
- 2 replies
- 2 kudos
- 92626 Views
- 2 replies
- 2 kudos
- 2 kudos
Z-Ordering is a technique to colocate related information in the same set of files. This co-locality is automatically used by Delta Lake on Databricks data-skipping algorithms to dramatically reduce the amount of data that needs to be read. Syntax fo...
- 2 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 586 | |
| 349 | |
| 287 |
