Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
I attended the Advanced Machine Learning with Databricks training last week virtually I am still unable to get the day 2 session videos of any of the Instructor led Paid Trainings. They are supposed to be available for replay with in 24 hours but I ...
During Job run, am trying to convert the dashboard to HTML form. Images in notebook are displayed as blank. am able to see the images displayed in Result only or Dashboard mode. Only during export to HTML, there is an issue with images. Could you p...
Hi @Siva K,Just a friendly follow-up. Did you have time to check Atanu's docs? did it help to resolve your question? please let us know if you need any help.
Hey there @Shakti Chand Hope all is well! Just wanted to check in if you were able to resolve your issue, and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...
I am using init scripts and would like to be able to control the version of a component that we release internally and frequently. We are now manually updating a dbfs requirement.txt file but I think that this problem may have been encountered befor...
You can programmatically create cluster templates in JSON files and include config JSON files with libraries needed. Cluster deployment in that scenario needs to be controlled via API https://docs.databricks.com/dev-tools/api/latest/clusters.html
Use Case Sharing Sweepstakes ! Data + AI Summit is in full swing and we know you are just as excited as we are to learn about the new and exciting things happening at Databricks. From notebooks to the Lakehouse, we know some of these new features wil...
Cloning libraries when cloning clustersCurrently when we clone clusters, the externally added libraries aren't copied as part of cloning process.It's an expected behavior but a missing one. At times new developers end up spending lot of time in debug...
For general cluster decision making refer to this article https://docs.microsoft.com/en-gb/azure/databricks/clusters/cluster-config-best-practicesOnce you've selected a cluster that makes sense, run it and check your ganglia metrics to see whether yo...
My favorite feature announced at Data+AI Summit is Connect from everywhere. I have been dreaming for a long time about sending SQL queries to SQL endpoint via API/SDK
It’s just a breeze for all the streaming users. What’s the best venue to learn more about it. Is there a Jira ticket that tracks all the progresses? also wonder which Spark version it will come with.
There is support for running on GPU which will be beneficial to certain ML workloads.Clusters are configured to run on CPU by default, but you can choose GPU based nodes during creation.
