- 16378 Views
- 5 replies
- 2 kudos
Is it possible to use multiprocessing or threads to submit multiple queries to a database from Databricks in parallel?
We are trying to improve our overall runtime by running queries in parallel using either multiprocessing or threads. What I am seeing though is that when the function that runs this code is run on a separate process it doesnt return a dataFrame with...
- 16378 Views
- 5 replies
- 2 kudos
- 2 kudos
Thanks for the links mentioned above. But both of them uses raw python to achieve parallelism. Does this mean Spark (read PySpark) does exactly provisions for parallel execution of functions or even notebooks ? We used a wrapper notebook with ThreadP...
- 2 kudos
- 528 Views
- 0 replies
- 0 kudos
Our Python Code Examples covers basic concepts, control structures, functions, lists, classes, objects, inheritance, polymorphism, file operations, da...
Our Python Code Examples covers basic concepts, control structures, functions, lists, classes, objects, inheritance, polymorphism, file operations, data structures, sorting algorithms, mathematical functions, mathematical sequences, threads, exceptio...
- 528 Views
- 0 replies
- 0 kudos
- 8309 Views
- 3 replies
- 1 kudos
Resolved! Plotting in pyspark.pandas Uncaught ReferenceError Plotly is not defined
Hi, I am trying to plot using pyspark.pandas running this sample code: speed = [0.1, 17.5, 40, 48, 52, 69, 88] lifespan = [2, 8, 70, 1.5, 25, 12, 28] index = ['snail', 'pig', 'elephant', 'rabbit', 'giraffe', 'coyote', 'horse'] psdf = ps.Data...
- 8309 Views
- 3 replies
- 1 kudos
- 1 kudos
Thank you @Werner Stinckens​ . I was able to find the plotly documentation listed below and setting the output_type and calling displayHTML() helped remedy the error.
- 1 kudos