cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

timothy_uk
by New Contributor III
  • 754 Views
  • 1 replies
  • 1 kudos

Mysterious simultaneous long-running Databricks Workflows

Hi,This happened across 4x seemingly unrelated workflows at the same time of the day - all 4x workflows eventually completed successfully. It appeared that all workflows sat idling despite triggering via the Jobs API. The two symptoms I have observed...

  • 754 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Timothy Lin​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 1 kudos
psps
by New Contributor III
  • 2985 Views
  • 3 replies
  • 4 kudos

Databricks Job run logs only shows prints/logs from driver and not executors

Hi,​In Databricks Job run output, only logs from driver are displayed. We have a function parallelized to run on executor nodes. The logs/prints from that function are not displayed in job run output. Is there a way to configure and show those logs i...

  • 2985 Views
  • 3 replies
  • 4 kudos
Latest Reply
psps
New Contributor III
  • 4 kudos

Thanks @Debayan Mukherjee​ . This is to enable executor logging. However, the executor logs do not appear in Databricks Job run output. Only driver logs are displayed.

  • 4 kudos
2 More Replies
pvignesh92
by Honored Contributor
  • 776 Views
  • 1 replies
  • 0 kudos

Resolved! How to use caller's rights in Databricks Workflows?

I have created a job in Databricks Workflow that runs a notebook and have given manage run permissions to users. If any user trigger the job, I expect the job to show their username in Run As Option as it was triggered by them and they have Manage Ru...

  • 776 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Vigneshraja Palaniraj​ Rus as is a private preview feature. Please complete this sign up form which will be reviewed by PM and you'll receive the confirmation once it's enabled.

  • 0 kudos
Netty
by New Contributor III
  • 2630 Views
  • 1 replies
  • 2 kudos

What's the crontab notation for every other week for Databricks Workflow scheduling?

Hello,I need to schedule some of my jobs within Databricks Workflow every other week (or every 4 weeks). I've scoured a few forums for find what this notation would be, but I've been unfruitful in my searches.Is this scheduling possible in crontab? I...

  • 2630 Views
  • 1 replies
  • 2 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 2 kudos

For every seven days starting from Monday, you need to use 2/7. From my experience, that generator works best with databricks https://www.freeformatter.com/cron-expression-generator-quartz.html 

  • 2 kudos
Labels