tqdm progressbar in Databricks jobs
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-20-2024 09:37 AM
Hi,
I'm using Databricks workflows to run a train job using `pytorch` + `lightning`. `lightning` has a built in progressbar built on `tqdm` that tracks the progress. It works OK in when I run the notebook outside of a workflow. But when I try to run notebook as a part of a workflow no progressbar is displayed.
Is this some limitation of the workflows or am I hitting some kind of bug?
I'm on Databricks runtime for ML 15.3 with `tqdm==4.65.0
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-29-2024 07:46 AM
I'm having the same issue. I even tried following
from tqdm.autonotebook import tqdm
but when I do this
for row in tqdm(df.itertuples(index=False)):
I don't see the progress bar in Databricks workflow notebook.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-24-2025 12:58 PM
I have found that only progressbar2 can work in both interactive notebooks and workflow notebooks. It's limited, but better than nothing. Tqdm is broken in workflows.
