cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks Cluster Web terminal different permissions with tmux and xterm.

ae20cg
New Contributor III

I am launching web terminal on my databricks cluster and when I am using the ephemeral xterm instance I am easily able to navigate to desired directory in `Workspace` and run anything... for example `ls ./`

When I switch to tmux so that I can preserve the shell instance, I navigate to the same directory and cannot access those same directories.

```

ls: cannot open directory '.' : Operation not permitted

```

I tried with sudo to see if for some reason this would be the issue but that did not solve it.

Any ideas?

5 REPLIES 5

Debayan
Databricks Employee
Databricks Employee

Hi @Andrej Erkelens​ , Could you please check the limitations here: https://docs.databricks.com/clusters/web-terminal.html#limitations also,

Databricks proxies the web terminal service from port 7681 on the cluster’s Spark driver. This web proxy is intended for use only with the web terminal. If the port is occupied when the cluster starts or if there is otherwise a conflict, the web terminal may not work as expected. If other web services are launched on port 7681, cluster users may be exposed to potential security exploits. Databricks is not responsible for any issues that result from the installation of unsupported software on a cluster.

Please let us know if this helps.

ae20cg
New Contributor III

@Debayan Mukherjee​ 

The port does not seem to be the issue. For some reason the permissions change when launching tmux vs xterm, and I cannot figure out why.

Anonymous
Not applicable

HI @Andrej Erkelens​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

alenka
New Contributor III

Hey there, fellow data explorer pals! I totally get your excitement when launching that web terminal on your Databricks cluster and feeling the power of running commands like 'ls ./' in the ephemeral xterm instance. It's like traversing the vast universe of data!

Now, I feel your frustration when switching to tmux and encountering that pesky error: 'ls: cannot open directory '.' : Operation not permitted'. But fear not, for I've faced a similar space oddity before.

My advice to you, dear author, is to check the permissions and ownership of the directories you're trying to access within tmux. Sometimes, tmux can carry over permissions from the initial shell instance, causing these roadblocks. Use 'ls -la' to check the permissions, and 'chown' or 'chmod' to adjust them if needed. Also, ensure that you're using the same user context for tmux and xterm.

In my case, after some tinkering, I had the tmux and xterm both singing in harmony, exploring the same directories effortlessly!

m997al
Contributor III

I have a very similar issue.  I can "cp" a file from "/Volumes/..." in Unity Catalog locally using xterm, but I cannot do the same thing (with the same root user) inside a tmux session.

No amount of "chown" or "chmod" has fixed this.  This is really a pain, as I have some pipelines that are nearly working inside the persistent tmux sessions.

Can you describe precisely what you did?  I get the error "cp: cannot stat '/Volumes/....': Operation not permitted" inside tmux, but it works fine in the xterm window.

Thanks!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group