topic Import module never ends on random branches in Data Engineering

Import module never ends on random branches

Valentin14 — Wed, 03 May 2023 15:30:43 GMT

Hello,

Since a week ago, our notebook are stuck in running on the firsts cells which import python module from our github repository which is cloned in databricks.

The cells stays in running state and when we try to manually cancel the jobs in databricks, it remains in cancelling and never end until the cluster is terminated.

What is really weird, is that we have the issue only on specific branches, and if we recreate the branch there is no more issue...

When the notebooks are submitted from adf, i got the following error(when it fails, some notebooks run ok, others doesn't) :

OSError: [Errno 5] Input/output error: '/Workspace/Repos/datafactory-view-develop/***/src/curated/business'

The workspaces are clones from github repo, so I don't understand why i have issues when loading module from our workspaces, when same module is loaded in another notebook without issue...

I have close to no logs describing the issue in the cluster logs so it's really hard to identify the cause of this issue..

The cluster runs in Azure environment.

Does anyone already encountered this kind of issue ?

Re: Import module never ends on random branches

Anonymous — Fri, 19 May 2023 07:49:41 GMT

Hi @Valentin LEBOSSE

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!

Re: Import module never ends on random branches

Debayan — Sat, 06 May 2023 06:06:05 GMT

Hi @Valentin Rosca, Without the logs it will be very hard to determine the cause of the issue. As the first step could you please raise a support case to Azure so that they can do the primary investigation on this and reach out to us if anything is needed.

Please tag @Debayan with your next comment so that I will get notified. Thank you!

Re: Import module never ends on random branches

mgradowski — Wed, 26 Jun 2024 12:43:31 GMT

Error from FUSE process during `ls /Workspace`:

2024/06/26 11:15:39.286089 workspace.WARNING [CP] ResponseLogHook: databricks api https://westeurope-c2.azuredatabricks.net/api/2.0/workspace-files/list-files: 504 Gateway Timeout. body:{"error_code":"TEMPORARILY_UNAVAILABLE","message":"The service at /api/2.0/workspace-files/list-files is taking too long to process your request. Please try again later or try a faster operation. [TraceId: 00-0a794387a987fc4de1cb8a6b70214774-97e61adb2ab644d5-00]"}

Re: Import module never ends on random branches

timo199 — Wed, 14 Aug 2024 05:58:37 GMT

How do you determine what permissions are required to access Fuse?

Please answer in detail

Re: Import module never ends on random branches

timo199 — Fri, 16 Aug 2024 00:12:05 GMT

@Retired_mod