Databricks Community

tawarity · ‎01-29-2024

Hi All,

I've been using notebooks to run patch requests to an external API using the Python requests library. Often times certain notebooks will randomly start to fail throughout the day and will raise a ImportHookFinder.find_spec() error when attempting the patch. Using identical code between notebooks this can fail in one notebook but still work on another all while using the same compute cluster. I was wondering if anyone has inclination on what this might be? I've tried switching compute run times and encounter the same error.

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 939, in _find_spec
AttributeError: 'ImportHookFinder' object has no attribute 'find_spec'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3508, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "/root/.ipykernel/979/command-1199240160521355-2663465150", line 6, in <module>
    domain_response = turn_domains_off(tactic_id, domain_list,tk)
  File "/root/.ipykernel/979/command-4074954368020290-3521920109", line 32, in turn_domains_off
    response = requests.patch(endpoint, headers=token_header, json=payload)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/requests/api.py", line 145, in patch
    return request("patch", url, data=data, **kwargs)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/requests/sessions.py", line 575, in request
    prep = self.prepare_request(req)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/requests/sessions.py", line 483, in prepare_request
    auth = get_netrc_auth(request.url)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/requests/utils.py", line 209, in get_netrc_auth
    from netrc import NetrcParseError, netrc
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 941, in _find_spec
  File "<frozen importlib._bootstrap>", line 914, in _find_spec_legacy
ImportWarning: ImportHookFinder.find_spec() not found; falling back to find_module()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 939, in _find_spec
AttributeError: 'ImportHookFinder' object has no attribute 'find_spec'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 2105, in showtraceback
    stb = self.InteractiveTB.structured_traceback(
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1428, in structured_traceback
    return FormattedTB.structured_traceback(
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1319, in structured_traceback
    return VerboseTB.structured_traceback(
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1172, in structured_traceback
    formatted_exception = self.format_exception_as_a_whole(etype, evalue, etb, number_of_lines_of_context,
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1062, in format_exception_as_a_whole
    self.get_records(etb, number_of_lines_of_context, tb_offset) if etb else []
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1113, in get_records
    style = get_style_by_name("default")
  File "/databricks/python/lib/python3.10/site-packages/pygments/styles/__init__.py", line 78, in get_style_by_name
    mod = __import__('pygments.styles.' + mod, None, None, [cls])
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 941, in _find_spec
  File "<frozen importlib._bootstrap>", line 914, in _find_spec_legacy
ImportWarning: ImportHookFinder.find_spec() not found; falling back to find_module()

ChrisLawford_n1 · a month ago

Hey,
Did you manage to solve this issue? I am experiencing the same error but under a different context of using pytest in a notebook.
My scenario:

import pytest
import os
import sys

os.chdir(f'/Workspace/Shared/DataTechnology-Databricks-Core')
# Skip writing pyc files on a readonly filesystem.
sys.dont_write_bytecode = True
    
# Run pytest.
retcode = pytest.main([".", "-v", "-p", "no:cacheprovider", " --import-mode=importlib"])
# Fail the cell execution if there are any test failures.
assert retcode == 0, "The pytest invocation failed. See the log for details."

And when I run it I get the error:

ImportError while loading conftest '/Workspace/Shared/DataTechnology-Databricks-Core/tests/conftest.py'.
AttributeError: 'ImportHookFinder' object has no attribute 'find_spec'
AssertionError: The pytest invocation failed. See the log for details.
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
File <command-2886421435072391>, line 12
     10 retcode = pytest.main([".", "-v", "-p", "no:cacheprovider", " --import-mode=importlib"])
     11 # Fail the cell execution if there are any test failures.
---> 12 assert retcode == 0, "The pytest invocation failed. See the log for details."

AssertionError: The pytest invocation failed. See the log for details.