Can not access a sql file from Notebook

[nyehia]

Hey,

I have a repo of notebooks and SQL files, the typical way is to update/create notebooks in the repo then push it and CICD pipeline deploys the notebooks to the Shared workspace.

the issue is that I can access the SQL files in the Repo but can not read them in the Shared workspace, however whenever I do ls or dbutils.fs.ls I can see the SQL files(without seeing the extension in the name) but can not read them.

Error is (Operation not supported)

 

Thanks

[Anonymous]

@Nermin Yehia​ :

It's possible that the SQL files are not being copied over to the Shared workspace during the deployment process. Double-check that the CICD pipeline is configured to include the SQL files in the deployment.

Another possible issue could be with the permissions on the SQL files. Check that the user who is trying to access the SQL files has the necessary permissions to read them. You can try changing the permissions using the chmod command in a notebook cell:

%sh
chmod 644 /dbfs/path/to/sql/file.sql

This will set the permissions on the SQL file to read-only for the owner, and read-only for everyone else. You can also try setting the permissions to read-write for the owner:

%sh
chmod 600 /dbfs/path/to/sql/file.sql

If none of these solutions work, it's possible that there is a more complex issue with the Shared workspace or the deployment process.

[nyehia]

Hey, Thank you so much for your help.

I can see the files are copied to the shared folder, I can list them but can not read them.

I tried to run the cmod command but got the same error

OSError: [Errno 95] Operation not supported:

[nyehia]

@Suteja Kanuri​ @Vidula Khanna​  please check my update. Thanks

[Anonymous]

@Nermin Yehia​ :

The error "OSError: [Errno 95] Operation not supported" typically occurs when you are trying to change the permissions on a file or directory that is located on a network file system (NFS) share or a shared folder that does not support the requested operation.

In your case, it seems that you are trying to read files from a shared folder that you can list but cannot read, and you are also unable to change the permissions using the chmod command.

One possible solution is to check the permissions and ownership of the files on the shared folder. Make sure that the permissions are set correctly and that you have the necessary permissions to read the files. You may also need to check the ownership of the files and ensure that you have the necessary privileges to access them.

Another possible solution is to check the configuration of the network file system or shared folder that you are using. Some network file systems or shared folders may have limitations or restrictions on the types of operations that can be performed on them. You may need to consult with the system administrator or IT team responsible for the shared folder to determine if there are any restrictions or limitations that may be causing the issue.

[nyehia]

Hey @Suteja Kanuri​ Thank you so much for your help.

actually, I tried to get the difference in permissions

also, I tried to set the permissions in the /Workspace/Shared/...sql file the same as Repo permissions but still getting the same error

[Anonymous]

Hi @Nermin Yehia​ 

Hope everything is going great.

Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you. 

Cheers!

[nyehia]

Sorry for the late reply. I replied now after testing the suggested solution but it is still not working.

[TPSteve]

We received the same error when our CI/CD pipeline deployed files from the repository to the workspace. The issue was that the imported files were of type "notebook" and not type "file". When we deleted the notebooks from the workspace and re-imported them as files, we no longer received the error. Take a look at your CI/CD pipeline, our DevOps engineer said the file types were intentionally being stripped from the files prior to upload. We're investigating updating the pipeline to not do so.

[ok_1]

ok