Databricks

del1000 · 05-29-2023

I have a code:from time import sleep from random import random from operator import add def f(a: int) -> float: sleep(0.1) return random() rdd1 = sc.parallelize(range(20), 2) rdd2 = sc.parallelize(range(20), 2) rdd3 = sc.parallelize(rang...

del1000 · 09-14-2021

Scenario:I tried to run notebook_primary as a job with same parameters' map. This notebook is orchestrator for notebooks_sec_1, notebooks_sec_2, and notebooks_sec_3 and next. I run them by dbutils.notebook.run(path, timeout, arguments) function.So ho...

del1000 · 10-03-2021

@Balbir Singh , I'm newbie in Databricks but the manual says you can use a python cell and transfer variables to scala's cell by temp tables.https://docs.databricks.com/notebooks/notebook-workflows.html#pass-structured-data

del1000 · 09-15-2021

Thank you, Dan. That is exactly what I wanted  By the way, where are these property and method of dbutils described? I can't find any references to them.

del1000 · 09-14-2021

I join to post's thread. I'd like to have the same but in the notebooks, not by CLI:Is it possible to passthrough job's parameters to variable? (databricks.com)

del1000 · 09-14-2021

Sorry but it is not the answer for my question.In notebook_primary, I don't know all names of node of arguments. I'd like to iterate arguments, do some change within it and send to process to each notebook_sec_* at all.The question is: is the noteboo...

Databricks

User Stats

User Activity

Problem with sparkContext.parallelize and volatile functions?

Is it possible to passthrough job's parameters to variable?

Re: Is it possible to passthrough job's parameters to variable?

Re: Is it possible to passthrough job's parameters to variable?

Re: How can I get the json spec of my Databricks Job?

Re: Is it possible to passthrough job's parameters to variable?