- 1081 Views
- 0 replies
- 0 kudos
Can you provide an example of what exactly do you mean? If the reference is to how "Repos" shows up in the UI, that's more for a Ux convenience. Repos as such are designed to be a container for version controlled notebooks that live in the Git reposi...
Delta Lake on Databricks has added runtime optimizations of the Delta Engine that further enhance the performance and scale of the open source Delta Format. In additional you also get access to a whole host of capabilities available on the Databricks...
Yes, you can pass parameters from ADF —> Azure Databricks.https://docs.microsoft.com/en-us/azure/data-factory/solution-template-databricks-notebook#how-to-use-this-templateYou can also pass values back from the Notebook --> ADF via the dbutils.notebo...
The import statements work fine with maven libraries freshly installed on a running cluster and on restart of the same cluster, it fails to import the classes.
When you install maven libraries one by one in running clusters, it is resolved and gets downloaded individually. The behavior would be different when you restart the clusters since the libraries are resolved together and can run into issues due to c...
The cluster will be terminated. But anyone with CAN_MANAGE can continue to start it, or clone it, and then delete it.
For an optimal processing experience Databricks segregates the Spark application traffic from the management traffic to avoid Network contention. Spark application traffic communications between the Driver-Executor and the Executors themselves where ...
What could be the root cause
I never ran VACUUM on the Delta table. Will Delta perform direct listing on those directories, I am afraid the query performance is going to get degraded over timeHow about the log directories. I have more than 100k JSON files in the log directory
For both Data and logs, Delta does not perform listing. The transaction logs have the details of the files and the objects are directly accessed. Similarly with in the Delta log directory listing is performed. However, it's performed using a continu...
30 mins. So, for example, via CloudTrail you might see a call every 30 minutes, but depending on how the notebooks are being accessed.
I would like to move my saved experiments and artifacts to a different Databricks workspace from where I originally created them. Is this possible?
It might be possible with a bit of code via mlflow client api ( there seems to be a way to run list_registered_models and extract info ) - but haven't tried it out. If the requirement is to share models between workspaces, one approach could be to h...
Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!
Sign Up NowUser | Count |
---|---|
1614 | |
773 | |
349 | |
286 | |
253 |