- 3597 Views
- 1 replies
- 3 kudos
Hello guys,I'm trying to migrate a python project from Pandas to Pandas API on Spark, on Azure Databricks using MLFlow on a conda env.The thing is I'm getting the next error:Traceback (most recent call last): File "/databricks/mlflow/projects/x/data_...
- 3597 Views
- 1 replies
- 3 kudos
Latest Reply
it should be yes.can you elaborate on how you create your notebook (and the conda env you talk about)?
by
Vik1
• New Contributor II
- 4194 Views
- 4 replies
- 2 kudos
My setup:Worker type: Standard_D32d_v4, 128 GB Memory, 32 Cores, Min Workers: 2, Max Workers: 8Driver type: Standard_D32ds_v4, 128 GB Memory, 32 CoresDatabricks Runtime Version: 10.2 ML (includes Apache Spark 3.2.0, Scala 2.12)I ran a snowflake quer...
- 4194 Views
- 4 replies
- 2 kudos
Latest Reply
Hey there @Vivek Ranjan Checking in. If Joseph's answer helped, would you let us know and mark the answer as best? It would be really helpful for the other members to find the solution more quickly.Thanks!
3 More Replies
by
Dan_Z
• Databricks Employee
- 750 Views
- 0 replies
- 0 kudos
mapInPandas is one of the most powerful Spark functions. It uses an arrow-like in-memory data structure to split up Spark Data Frames into chunks and feeding them to a function that takes a Pandas DF as input and output. Check it out here:https://spa...
- 750 Views
- 0 replies
- 0 kudos
