Databricks Community

jeremy98

Hi community,The if else condition task couldn't be used as real if condition? Seems that if the condition goes to False the entire job will be stop. Is it a right behaviour?

jeremy98

Hi community,I was reading the Databricks API documentation and I want to get information about one job if this is schedule with the status PAUSED or UNPAUSED. I was watching that there is this api call: https://docs.databricks.com/api/workspace/jobs...

jeremy98

Hello community,I was working on optimising the driver memory, since there are code that are not optimised for spark, and I was planning temporary to restart the cluster to free up the memory.that could be a potential solution, since if the cluster i...

jeremy98

Hi community,We have deployed the wheel package internally in our bundle repository: artifacts: rnc_lib: type: whl build: poetry build path: . # For passing wheel package to workspace sync: include: - ./dist/*.whlThe problem is t...

jeremy98

Hi community,Is there a way to install dependencies inside a notebook task using serveless compute using Databricks Asset Bundle? Is there a way to avoid installing everytime for each serverless task that compose a job the dependencies (or the librar...

jeremy98

Hi, I found that the problem is here: - task_key: get_email_infos max_retries: 3 min_retry_interval_millis: 150000 depends_on: - task_key: check_type_of_trigger outcome: "true" ...

jeremy98

Hi dmy,Yes more general not only cluster configurations! But, we have created a custom example where setting this and it's working fine :). Btw, thanks Koji! Thanks all 

jeremy98

Yes, thanks, the problem was the access_token that is not set in the other environment 

jeremy98

any suggestion Mr. @Alberto_Umana ?

jeremy98

Hi,The code synchronizes Databricks with PostgreSQL by identifying differences and applying INSERT, UPDATE, or DELETE operations to update PostgreSQL. The steps are as follows:Read the source data in Databricks using a simple spark.sql query.Read the...

Databricks Community

User Stats

User Activity

if else condition task doubt

how to get schedule information about a job in databricks?

restarting the cluster always running doesn't free the memory?

where saving the wheel package?

Environment set up in serveless notebook task

Re: if else condition task doubt

Re: Best practice on how to create a configuration yaml files for each workspace environment based?

Re: how to get schedule information about a job in databricks?

Re: restarting the cluster always running doesn't free the memory?

Re: restarting the cluster always running doesn't free the memory?