Python Requests Library Error ImportHookFinder.find_spec()

[tawarity]

Hi All,

I've been using notebooks to run patch requests to an external API using the Python requests library. Often times certain notebooks will randomly start to fail throughout the day and will raise a ImportHookFinder.find_spec() error when attempting the patch. Using identical code between notebooks this can fail in one notebook but still work on another all while using the same compute cluster. I was wondering if anyone has inclination on what this might be? I've tried switching compute run times and encounter the same error.

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 939, in _find_spec
AttributeError: 'ImportHookFinder' object has no attribute 'find_spec'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3508, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "/root/.ipykernel/979/command-1199240160521355-2663465150", line 6, in <module>
    domain_response = turn_domains_off(tactic_id, domain_list,tk)
  File "/root/.ipykernel/979/command-4074954368020290-3521920109", line 32, in turn_domains_off
    response = requests.patch(endpoint, headers=token_header, json=payload)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/requests/api.py", line 145, in patch
    return request("patch", url, data=data, **kwargs)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/requests/sessions.py", line 575, in request
    prep = self.prepare_request(req)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/requests/sessions.py", line 483, in prepare_request
    auth = get_netrc_auth(request.url)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/requests/utils.py", line 209, in get_netrc_auth
    from netrc import NetrcParseError, netrc
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 941, in _find_spec
  File "<frozen importlib._bootstrap>", line 914, in _find_spec_legacy
ImportWarning: ImportHookFinder.find_spec() not found; falling back to find_module()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 939, in _find_spec
AttributeError: 'ImportHookFinder' object has no attribute 'find_spec'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 2105, in showtraceback
    stb = self.InteractiveTB.structured_traceback(
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1428, in structured_traceback
    return FormattedTB.structured_traceback(
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1319, in structured_traceback
    return VerboseTB.structured_traceback(
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1172, in structured_traceback
    formatted_exception = self.format_exception_as_a_whole(etype, evalue, etb, number_of_lines_of_context,
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1062, in format_exception_as_a_whole
    self.get_records(etb, number_of_lines_of_context, tb_offset) if etb else []
  File "/databricks/python/lib/python3.10/site-packages/IPython/core/ultratb.py", line 1113, in get_records
    style = get_style_by_name("default")
  File "/databricks/python/lib/python3.10/site-packages/pygments/styles/__init__.py", line 78, in get_style_by_name
    mod = __import__('pygments.styles.' + mod, None, None, [cls])
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 941, in _find_spec
  File "<frozen importlib._bootstrap>", line 914, in _find_spec_legacy
ImportWarning: ImportHookFinder.find_spec() not found; falling back to find_module()

[Kaniz]

Hi @tawarity, There are a few possible causes for this issue:

1. Inconsistent Environment: The error might be due to inconsistencies in the environment across different notebooks. Even though the notebooks are running on the same cluster, there might be differences in the environ...

2. Databricks Repos Feature: If you’re using the Databricks Repos feature to import notebooks as module.... One user reported a similar issue and found that the problem was due to using an incorrect Databrick...

3. Importing Notebooks as Modules: You might encounter issues if you’re trying to import a Databricks Notebook as a Python module. According to the Azure Databricks documentation, if you want to import a notebook as a Python module, you need to edit the notebook in a code editor and remove the line # Databricks Notebook source. Removing this line converts the notebook to a regular Python file.

To resolve this issue, you could try the following:

• Ensure that the environments of all your notebooks are consistent.
• Check the DBR version you’re using and ensure it’s compatible with the features you’re using.
• If you’re importing notebooks as modules, make sure to follow the correct procedure as outlined in t...

I hope this helps! Let me know if you have any other questions.

[Kaniz]

Hey there! Thanks a bunch for being part of our awesome community! 🎉 

We love having you around and appreciate all your questions. Take a moment to check out the responses – you'll find some great info. Your input is valuable, so pick the best solution for you. And remember, if you ever need more help , we're here for you! 

Keep being awesome! 😊🚀