cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

horatiug
by New Contributor III
  • 496 Views
  • 0 replies
  • 0 kudos

Can the databricks_mount timeout be changed. ?

I am using terrafom to do databricks workspace configuration and while mounting 6 buckets if duration of mount is bigger than 20 min I get timeout. Is it possible to change the timeout ? thanksHoratiu

  • 496 Views
  • 0 replies
  • 0 kudos
augustin
by New Contributor II
  • 3087 Views
  • 5 replies
  • 5 kudos

Mount an uncrypted AWS EFS in AWS Databricks

Hi,I want to mount an uncrypted AWS EFS in AWS Databricks. When I do:mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-abcdef.efs.region.amazonaws.com:/ /mnt/efs-uncryptedI get this error:mount.nfs4: moun...

  • 3087 Views
  • 5 replies
  • 5 kudos
Latest Reply
Andrei_Radulesc
Contributor III
  • 5 kudos

"To support NFS under LXC, some of the apparmor protections need to be lifted." (see https://theorangeone.net/posts/mount-nfs-inside-lxc/)

  • 5 kudos
4 More Replies
LukaszJ
by Contributor III
  • 2229 Views
  • 5 replies
  • 4 kudos

Resolved! Mount Azure Blob Storage with Cluster access control

Hello.I want to mount and share for the one group the container from Azure Blob Storage (It could be simple blob storage or Azure Data Lake Storage gen 2). But I am not able to do it because I am using Cluster with Table Access Control.This is my cod...

  • 2229 Views
  • 5 replies
  • 4 kudos
Latest Reply
LukaszJ
Contributor III
  • 4 kudos

I have a good solution to the problem:I am using Python library.There are some documentation.Topic to be closed.Best regards,Łukasz

  • 4 kudos
4 More Replies
kkumar
by New Contributor III
  • 870 Views
  • 2 replies
  • 2 kudos

ADLS Gen 2 Delta Tables memory allocation

if i mount a gen2(ADLS 1) to another gen2(ADLS2) account and create a delta table on ADLS2 will it copy the data or just create something link External table.i don't want to duplicate the the data.

  • 870 Views
  • 2 replies
  • 2 kudos
Latest Reply
Pat
Honored Contributor III
  • 2 kudos

Hi @keerthi kumar​ ,so basically you can CREATE EXTERNAL TABLES on top of the data stored somewhere - in your case ADLS. Data won't be copied, it will stay where it is, by creating external tables you are actually storing the metadata in your metasto...

  • 2 kudos
1 More Replies
StuartParker188
by New Contributor III
  • 4379 Views
  • 5 replies
  • 5 kudos

How to handle java.io.Exception in python notebook

I'm attempting to mount a volume using dbutils.fs.mount in a python workbookin the exception handling for this statement, I have found an exception that doesn't get caught using the standard try/except handlingfor example, if passing through a contai...

  • 4379 Views
  • 5 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

Hi @Stuart Parker​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 5 kudos
4 More Replies
niels
by New Contributor III
  • 951 Views
  • 2 replies
  • 0 kudos

Azure SA mounted but can't load files

I am attempting to load an excel file that's located in a blob storage that I've mounted. In the first cell, when I use the dbutils.fs.ls command, I can see the file I want to load. However, when I try to actually load it, it can't find the file. It ...

  • 951 Views
  • 2 replies
  • 0 kudos
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hi @Niels Ota​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 0 kudos
1 More Replies
Bhanu1
by New Contributor III
  • 2887 Views
  • 4 replies
  • 6 kudos

Resolved! Is it possible to mount different Azure Storage Accounts for different clusters in the same workspace?

We have a development and a production data lake. Is it possible to have a production or development cluster access only respective mounts using init scripts?

  • 2887 Views
  • 4 replies
  • 6 kudos
Latest Reply
Kaniz
Community Manager
  • 6 kudos

Hi @Bhanu Patlolla​ ​, We haven’t heard from you on the last response from @Werner Stinckens​ and @Hubert Dudek​  and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be h...

  • 6 kudos
3 More Replies
rahul3
by New Contributor
  • 1803 Views
  • 2 replies
  • 1 kudos

Facing mount/unmount issue while running same job parallelly with scala.

 Using above configuration in cluster, when I run databricks job parallelly with multiple request at a same time, then I am getting mount/unmount issue. For an example : When I make three request to databricks job , it run 3 jobs parallelly but somet...

image.png image.png
  • 1803 Views
  • 2 replies
  • 1 kudos
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hi @rahul upadhyay​, We haven’t heard from you on the last response from @Prabakar Ammeappin​ . If you have any solution, please share it with the community as it can be helpful to others. Otherwise, we will respond with more details and try to help.

  • 1 kudos
1 More Replies
Ashley1
by Contributor
  • 1350 Views
  • 5 replies
  • 1 kudos

Resolved! Can ADLS be mounted in DBFS using only ADLS account key?

I realise this is not an optimal configuration but I'm trying to pull together a POC and I'm not at the point that I wish to ask the AAD admins to create an application for OAuth authentication.I have been able to use direct references to the ADLS co...

  • 1350 Views
  • 5 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hey there @Ashley Betts​ Thank you for posting your question. And you found the solution.This is awesome!Would you be happy to mark the answer as best so that other members can find the solution more quickly?Cheers!

  • 1 kudos
4 More Replies
PraveenKumar188
by New Contributor
  • 1904 Views
  • 4 replies
  • 2 kudos

Resolved! Is is possible to Mount multiple ADLS Gen2 Storage paths in single workspace

Hello Experts,We are looking on feasibility of mounting more that one ADLS Gen2 storages on a single workspace of databricks.Best RegardsPraveen

  • 1904 Views
  • 4 replies
  • 2 kudos
Latest Reply
Erik
Valued Contributor II
  • 2 kudos

Yes, its possible, we are doing it. Just mount them to different folders like @Werner Stinckens​ is saying.

  • 2 kudos
3 More Replies
matt_t
by New Contributor
  • 1994 Views
  • 3 replies
  • 1 kudos

Resolved! S3 sync from bucket to a mounted bucket causing a "[Errno 95] Operation not supported" error for some but not all files

Trying to sync one folder from an external s3 bucket to a folder on a mounted S3 bucket and running some simple code on databricks to accomplish this. Data is a bunch of CSVs and PSVs.The only problem is some of the files are giving this error that t...

  • 1994 Views
  • 3 replies
  • 1 kudos
Latest Reply
Atanu
Esteemed Contributor
  • 1 kudos

@Matthew Tribby​  does above suggestion work. Please let us know if you need further help on this. Thanks.

  • 1 kudos
2 More Replies
Confused
by New Contributor III
  • 2389 Views
  • 6 replies
  • 1 kudos

Hi Guys Is there any documentation on where the /databricks-datasets/ mount is actually served from?We are looking at locking down where our workspace...

Hi GuysIs there any documentation on where the /databricks-datasets/ mount is actually served from?We are looking at locking down where our workspace can reach out to via the internet and as it currently stands we are unable to reach this.I did look ...

  • 2389 Views
  • 6 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hello Mat, Thanks for letting us know. Would you be happy to mark your answer as best if that will solve the problem for others? That way, members will be able to find the solution more easily.

  • 1 kudos
5 More Replies
Prabakar
by Esteemed Contributor III
  • 1359 Views
  • 0 replies
  • 2 kudos

Accessing the regions that are disabled by default in AWS from Databricks. In AWS we have 4 regions that are disabled by default. You must first enabl...

Accessing the regions that are disabled by default in AWS from Databricks.In AWS we have 4 regions that are disabled by default. You must first enable it before you can create and manage resources. The following Regions are disabled by default:Africa...

  • 1359 Views
  • 0 replies
  • 2 kudos
Erik
by Valued Contributor II
  • 2744 Views
  • 8 replies
  • 2 kudos

Run more than nr-of-cores concurrent tasks.

We are using the terraform databricks provier, which is starting a cluster and checking every mount (since there is no mount rest API!). Each mount takes 20 seconds to check, and 99.9% of that time is idle waiting, and it starts a job per mount. If w...

  • 2744 Views
  • 8 replies
  • 2 kudos
Latest Reply
jose_gonzalez
Moderator
  • 2 kudos

hi @Erik Parmann​ ,It is possible to do, but you might need to also enable dynamic allocation at the cluster level to be able to make sure your settings are apply at cluster creation . You can find more details here. As best practice, we do not recom...

  • 2 kudos
7 More Replies
stramzik
by New Contributor II
  • 972 Views
  • 1 replies
  • 1 kudos

Unable to mount datalake gen1 to databricks

I was mounting the Datalake Gen1 to Databricks for accessing and processing files, The below code was working great for the past 1 year and all of a sudden I'm getting an errorconfigs = {"df.adl.oauth2.access.token.provider.type": "ClientCredential"...

  • 972 Views
  • 1 replies
  • 1 kudos
Latest Reply
stramzik
New Contributor II
  • 1 kudos

bumping up the thread

  • 1 kudos
Labels