cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

TheDataDexter
by New Contributor III
  • 2513 Views
  • 4 replies
  • 3 kudos

Resolved! Single-Node cluster works but Multi-Node clusters do not read data.

I am currently working with a VNET injected databricks workspace. At the moment I have mounted a the databricks cluster on an ADLS G2 resource. When running notebooks on a single node that read, transform, and write data we do not encounter any probl...

  • 2513 Views
  • 4 replies
  • 3 kudos
Latest Reply
ellafj
New Contributor II
  • 3 kudos

@TheDataDexter Did you find a solution to your problem? I am facing the same issue

  • 3 kudos
3 More Replies
iarregui
by New Contributor
  • 1130 Views
  • 2 replies
  • 0 kudos

Getting a Databricks static IP

Hello. I want to connect from my Databricks workspace to an external API to extract some data. The owner of the API asks for an IP to provide the token necessary for the extraction of data. Therefore I would need to set a static IP in Databricks that...

  • 1130 Views
  • 2 replies
  • 0 kudos
Latest Reply
Wojciech_BUK
Contributor III
  • 0 kudos

Hello, the easiest way (in Azure) is to deploy Workspace in VNET Injection mode and attach NAT Gateway to you VNET. NAT GW require Public IP. This IP will be your static egress IP for all Cluster in for this Workspace.Note: both NAT GW and IP Address...

  • 0 kudos
1 More Replies
Dean_Lovelace
by New Contributor III
  • 10830 Views
  • 10 replies
  • 2 kudos

How can I deploy workflow jobs to another databricks workspace?

I have created a number of workflows in the Databricks UI. I now need to deploy them to a different workspace.How can I do that?Code can be deployed via Git, but the job definitions are stored in the workspace only.

  • 10830 Views
  • 10 replies
  • 2 kudos
Latest Reply
Radhikad
New Contributor II
  • 2 kudos

Hello everyone, I need the same help from databricks expert. I have created a 'Job1' job with runtime 12.2 in 'Datbricks1' workspace. I have integrated with Azure repo and tried deploying in 'ENV1' using CI/CD pipeline. It is successfully deployed in...

  • 2 kudos
9 More Replies
User16752245312
by New Contributor III
  • 1698 Views
  • 3 replies
  • 2 kudos
  • 1698 Views
  • 3 replies
  • 2 kudos
Latest Reply
Aria
New Contributor III
  • 2 kudos

We are using azure.I dont see an option for deployment name. Secondly, we have already deployed all our workspaces and wants to have user friendly URLs.Like some changes in DNS server or proxy URLs.

  • 2 kudos
2 More Replies
ranged_coop
by Valued Contributor II
  • 10530 Views
  • 23 replies
  • 22 kudos

Resolved! How to access a jar file stored in Databricks Workspace ?

Hi All,We have a couple of jars stored in a workspace folder.We are using init scripts to copy the jars in the workspace to the /databricks/jars path.The init scripts do not seem to be able to find the files. The scripts are failing saying the files ...

  • 10530 Views
  • 23 replies
  • 22 kudos
Latest Reply
Anonymous
Not applicable
  • 22 kudos

Hi @Bharath Kumar Ramachandran​ You're welcome! I'm glad you found the link useful. I empathize with your hope that Databricks would consider adding this option. It's possible that Databricks will take user feedback into account when planning future ...

  • 22 kudos
22 More Replies
nistrate
by New Contributor II
  • 3194 Views
  • 2 replies
  • 3 kudos

Resolved! Revoking cluster creation permission for non-admin users in Databricks workspace

I'm currently building a Databricks workspace for my organization and I would like to revoke the cluster creation permission for all users except admins or a specific group (e.g., Data Analysts). Could someone please guide me on how to achieve this?

  • 3194 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Nicolae Istrate​ We haven't heard from you since the last response from @Werner Stinckens​ , and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful ...

  • 3 kudos
1 More Replies
rami1
by New Contributor II
  • 5140 Views
  • 2 replies
  • 3 kudos

METASTORE_DOWN: Cannot connect to metastore

I am trying to view databases and tables, default as well user created but it looks like the cluster created is not able to connect. I am using databricks default hive metastore. Viewing cluster logs provide following ventMETASTORE_DOWN Metastore is...

  • 5140 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

@rami​ :If the metastore is down, it means that the Databricks cluster is not able to connect to the metastore. Here are a few things you can try to resolve the issue:Check if the Hive metastore is up and running. You can try to connect to the metast...

  • 3 kudos
1 More Replies
joao_vnb
by New Contributor III
  • 1293 Views
  • 2 replies
  • 0 kudos
  • 1293 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @joao vnb​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we c...

  • 0 kudos
1 More Replies
Hari_Dbrc
by New Contributor II
  • 2247 Views
  • 2 replies
  • 0 kudos

Issue while using community edition

Hello,Is anyone facing issue with their community edition.?Shows the below error and cant access the workspace or previosly created notebooks..Tried accessing from different devices..(not a cache issue)Error popup:(screenshots attached)Unable to view...

  • 2247 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Hari N​ Thank you for reaching out, and we’re sorry to hear about this log-in issue! We have this Community Edition login troubleshooting post on Community. Please take a look, and follow the troubleshooting steps. If the steps do not resolve the...

  • 0 kudos
1 More Replies
Avvar2022
by Contributor
  • 1851 Views
  • 2 replies
  • 2 kudos

Resolved! I am new to data bricks. setting up Workspace for NON-prod environment Separate workspaces for DEV, QA or Just one work space for NON-prod ?

What i learned based on learning materials, documents, etc.. For data bricks it is a good practice to set up 1 non-prod workspace but separate clusters for Dev, QA, SIT, etc.Is it best practice to set up only 1 NON-PROD Workspace instead of separate ...

Databricks non-prod workspace set up options
  • 1851 Views
  • 2 replies
  • 2 kudos
Latest Reply
Avvar2022
Contributor
  • 2 kudos

Thank you. This helps.

  • 2 kudos
1 More Replies
Chris_Shehu
by Valued Contributor III
  • 1997 Views
  • 4 replies
  • 2 kudos

Resolved! No Explicit Deny for User security configurations at the group level?

Currently when you add new users to the Databricks workspace they get added to a "Users" group that has full access to the workspace. There should be a way to use group security to explicitly deny access to those same settings. This setting should ov...

image image image
  • 1997 Views
  • 4 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@dean james​ I am not sure about your case why you want to deny access to the group once you create it. Anyhow, we can use deacticate/activate an user using "2.0/preview/scim/v2/Users/{id}" rest API endpoint. We can also deactivate users that have no...

  • 2 kudos
3 More Replies
Dayaa
by New Contributor II
  • 1452 Views
  • 3 replies
  • 3 kudos

Resolved! Load data into Azure SQL Database from Azure Databricks ( restricted table not a whole workspace tables)

Hi ,I want to share limited tables in my databricks workspace and users will connect to my databricks through Azure Data factory and will load data into Azure SQL. Is this possible using Delta Sharing? Or any other method or tool?

  • 1452 Views
  • 3 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Dayananthan Marimuthu​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your...

  • 3 kudos
2 More Replies
Arunsundar
by New Contributor III
  • 1677 Views
  • 4 replies
  • 3 kudos

Automating the initial configuration of dbx

Hi Team,Good morning.As of now, for the deployment of our code to Databricks, dbx is configured providing the parameters such as cloud provider, git provider, etc., Say, I have a code repository in any one of the git providers. Can this process of co...

  • 1677 Views
  • 4 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Arunsundar Muthumanickam​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear fr...

  • 3 kudos
3 More Replies
Baumeister
by New Contributor II
  • 1875 Views
  • 2 replies
  • 0 kudos

Error when importing .dbc of a complete Workspace

I saved the content of an older Databricks Workspace by clicking on the Dropdown next to Workspace -> Export -> DBC Archive and saved it on my local machine.In a new Databricks Workspace, I now want to import That .DBC archive to restore the previous...

dbcerror
  • 1875 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Sebastian K​ :It looks like the error you are facing while importing the DBC archive could be due to the version incompatibility between the Databricks instance where you created the DBC archive and the one where you are trying to import it. Can you...

  • 0 kudos
1 More Replies
JordanYaker
by Contributor
  • 1274 Views
  • 3 replies
  • 3 kudos

Resolved! What is the maximum number of workspaces per account using Databricks on AWS?

I've been looking through the documentation and I swear this used to be listed somewhere, but for the life of me I can't find it anymore.

  • 1274 Views
  • 3 replies
  • 3 kudos
Latest Reply
JordanYaker
Contributor
  • 3 kudos

Thanks @Kaniz Fatma​ 

  • 3 kudos
2 More Replies
Labels