by
Vik1
• New Contributor II
- 2362 Views
- 4 replies
- 2 kudos
My setup:Worker type: Standard_D32d_v4, 128 GB Memory, 32 Cores, Min Workers: 2, Max Workers: 8Driver type: Standard_D32ds_v4, 128 GB Memory, 32 CoresDatabricks Runtime Version: 10.2 ML (includes Apache Spark 3.2.0, Scala 2.12)I ran a snowflake quer...
- 2362 Views
- 4 replies
- 2 kudos
Latest Reply
Hey there @Vivek Ranjan Checking in. If Joseph's answer helped, would you let us know and mark the answer as best? It would be really helpful for the other members to find the solution more quickly.Thanks!
3 More Replies
- 1014 Views
- 3 replies
- 3 kudos
I am following the Apache Spark™ Tutorial. When I finish the data set part and want to continue the machine learning part. I found the page is empty. The next section after machine learning is fine. So I guess there must be a url mismatching.The url ...
- 1014 Views
- 3 replies
- 3 kudos
Latest Reply
I clean the cookie and then the link recovers. So it is an issue about cookie.
2 More Replies
- 608 Views
- 0 replies
- 1 kudos
2021-09 webinar: Automating the ML Lifecycle With Databricks Machine Learning (Post 2 of 2)Thank you to everyone who joined! You can access the on-demand recording here and the code in this Github repo.We're sharing a subset of the questions asked an...
- 608 Views
- 0 replies
- 1 kudos
- 455 Views
- 0 replies
- 1 kudos
2021-09 webinar: Automating the ML Lifecycle With Databricks Machine Learning (post 1 of 2)Thank you to everyone who joined the Automating the ML Lifecycle With Databricks Machine Learning webinar! You can access the on-demand recording here and the ...
- 455 Views
- 0 replies
- 1 kudos
- 2010 Views
- 1 replies
- 0 kudos
I'm a data scientist creating versioned ML models. For compliance reasons, I need to be able to replicate the training data for each model version. I've seen that you can version datasets by using delta, but the default retention period is around 30 ...
- 2010 Views
- 1 replies
- 0 kudos
Latest Reply
Delta, as you mentioned has a feature to do time travel and by default, delta tables retain the commit history for 30 days. Operations on history of the table are parallel but will become more expensive as the log size increasesNow, in this case - s...
- 577 Views
- 1 replies
- 0 kudos
I have an NLP application that I build on my local machine using spacy and pandas, but now I would like to scale my application to a large production dataset and utilize the benefits of sparks distributed compute. How do I import and utilize a librar...
- 577 Views
- 1 replies
- 0 kudos
Latest Reply
It depends on what you mean, but if you're just trying to (say) tokenize and process data with spacy in parallel, then that's trivial. Write a 'pandas UDF' function that expresses how you want to transform data using spacy, in terms of a pandas DataF...
- 325 Views
- 0 replies
- 0 kudos
Databricks Certified Professional Data Scientist Does this exam require Databricks-specific or Spark-specific knowledge?No. Test-takers will be assessed on their understanding of the basics of machine learning and data science, how to complete each ...
- 325 Views
- 0 replies
- 0 kudos
- 1093 Views
- 2 replies
- 0 kudos
How do I efficiently read image data for a deep learning model?
- 1093 Views
- 2 replies
- 0 kudos
Latest Reply
Our documentation provides nice examples of preparing image data for training and inference.Training: See docs for AWS, Azure, GCPInference: See reference solution for AWS, Azure, GCP
1 More Replies
