topic Re: Disable spark transformations inside try except warning in Data Engineering

Disable spark transformations inside try except warning

Kayla — Thu, 30 Jul 2026 18:35:16 GMT

I'm wondering if there's a way to suppress certain errors? I have a lot of spammy yellow underlining from this one warning, and it doesn't seem that the warning behaves properly. Transformation inside try/except block is lazy, fine, I have actions on the dataframe ALSO in the try/except so this is irrelevant.

I'm also a little confused what it's actually coming from. It says it's from databricks-flint, which sounds like a linter, but searching it is giving me a time-series library?

Warning message:
Spark transformations inside try/except are lazy and will not raise exceptions. Move the triggering action (e.g., .count(), .show(), .collect()) inside the try block instead.databricks-flint(SCPAP005)

Re: Disable spark transformations inside try except warning

adnan_alvee — Thu, 30 Jul 2026 20:34:09 GMT

Can u try this in your toml file?

[tool.databricks.flint] ignore = ["SCPAP005"]

Re: Disable spark transformations inside try except warning

Kayla — Fri, 31 Jul 2026 18:04:41 GMT

So...I don't have one of those? Guessing you mean pyproject.toml which I just added with your suggestion, and no change.

Re: Disable spark transformations inside try except warning

masonreed11 — Sat, 01 Aug 2026 05:06:58 GMT

If your action (count(), collect(), show(), etc.) is already inside the try block, this sounds like a false positive. The warning is likely from a Databricks inspection rather than the Flint time-series library. Check if inspection SCPAP005 can be disabled or its severity changed in your IDE, and if not, consider reporting it with a small reproducible example so the maintainers can investigate.

Re: Disable spark transformations inside try except warning

Kayla — Sat, 01 Aug 2026 12:50:54 GMT

I'm seeing this in the Databricks web editor, not my own IDE environment.
I'd disable it if I knew how, but I can't find any documentation for what it's coming from. If you google SCPAP005, all you get is this post and CPAP machines. Likewise, I'd report it to the maintainers if I knew who they were.

Re: Disable spark transformations inside try except warning

MartFaber — Tue, 25 Aug 2026 10:03:42 GMT

Hi @Kayla,

I was wondering if this has already been patched in some way, as I'm facing the exact same error.

Kind regards, Mart Faber

Re: Disable spark transformations inside try except warning

munzewerfen — Tue, 25 Aug 2026 12:36:31 GMT

Interesting discussion! Handling Spark transformations inside try-except blocks can be tricky because of Spark’s lazy evaluation and when errors are actually triggered. Clear explanations of these behaviors are especially helpful for developers troubleshooting data pipelines and improving error handling. For a quick choice between two approaches, Münze werfen can be a simple and fun way to decide.