06-22-2023 02:26 PM
Hello,
I am experiencing issues with importing from utils repo the schema file I created.
this is the logic we use for all ingestion and all other schemas live in this repo utills/schemas
I am unable to access the file I created for a new ingestion pipeline
and get the following error:
OSError: [Errno 95] Operation not supported: '/Workspace/Repos/Connectors/Dev/utils/schemas/Comptroller.py'
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
File <command-2008379283730306>:10
7 from pyspark.sql import DataFrame as SparkDataFrame
8 # from utils.schemas import comptroller as schemas # does not allow me to import
9 #workaround
---> 10 from utils.schemas.Comptroller import *
File /databricks/python_shell/dbruntime/PythonPackageImportsInstrumentation/__init__.py:172, in _create_import_patch.<locals>.import_patch(name, globals, locals, fromlist, level)
167 thread_local._nest_level += 1
169 try:
170 # Import the desired module. If you’re seeing this while debugging a failed import,
171 # look at preceding stack frames for relevant error information.
--> 172 original_result = python_builtin_import(name, globals, locals, fromlist, level)
174 is_root_import = thread_local._nest_level == 1
175 # `level` represents the number of leading dots in a relative import statement.
176 # If it's zero, then this is an absolute import.
File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)
File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)
File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)
File <frozen importlib._bootstrap_external>:879, in exec_module(self, module)
File <frozen importlib._bootstrap_external>:1016, in get_code(self, fullname)
File <frozen importlib._bootstrap_external>:1073, in get_data(self, path)
OSError: [Errno 95] Operation not supported: '/Workspace/Repos/Connectors/Dev/utils/schemas/Comptroller.py'
I tried to import a different file from the same location and it worked /utils/schemas/differentflie.py
I see all the contents in that file.
the schema files are self explanatory and contain only schema for the tables being ingested.
any idea why this is happening is it a repo sync issue? a permissions issue?
really confused by it.
Also I am wondering what the difference in the icons mean.
I assumed it means that this file was created by me
I'm a databricks newbie
Thank you in advance.
06-23-2023 12:05 AM
Hi @Arbi nazar
Great to meet you, and thanks for your question!
Let's see if your peers in the community have an answer to your question. Thanks.
06-23-2023 12:26 AM
Hi, Could you please check if you can access the file from a notebook instead?
https://docs.databricks.com/files/index.html
Please tag @Debayan with your next update so that I will get notified.
06-23-2023 09:45 AM
@Debayan Mukherjee
Hello, thank you for your response.
please let me know if these are the correct commands to access the file from notebook
I can see the files in the repo folder
but I just noticed this.
the file I am trying to access the size is 0 but there is contents in there so it shouldn't be 0
Thank you
06-17-2024 08:34 AM
Hi there,
Have we found a solution for this problem? I'm facing the same issue as you 😞
14 hours ago
@Arb
Make sure the below is a file with extension .py and not a notebook. That should resolve the issue.
/Workspace/Repos/Connectors/Dev/utils/schemas/Comptroller.py'
Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!
Sign Up Now