cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

jeremy98
by Honored Contributor
  • 5486 Views
  • 33 replies
  • 5 kudos

Databricks to SFTP: Connection Fails Even with Whitelisted NAT Gateway IP

Hi community,I’m experiencing a strange issue with my connection from Databricks to an SFTP server.I provided them with an IP address created for Databricks via a NAT gateway, and that IP is whitelisted on their side. However, even though I have the ...

  • 5486 Views
  • 33 replies
  • 5 kudos
Latest Reply
Kenji_3000
New Contributor III
  • 5 kudos

Hi @szymon_dybczak , Thanks for the suggestion. I tried to attach the ip address to the NAT but it requires a NAT also to have an internet routing preference and Azure doesn't allow that. Quite a unique scenario indeed. We have created a workaround n...

  • 5 kudos
32 More Replies
Paweł_Janczyk
by New Contributor
  • 1904 Views
  • 7 replies
  • 1 kudos

missing github email address in commits

Hello everyone, I would like to configure github client on my databricks workspace. Especially an email address in not visible and is replace by my github user name. Do you know how I can do that? 

Pawe_Janczyk_0-1747049752792.png
  • 1904 Views
  • 7 replies
  • 1 kudos
Latest Reply
aakashnand-kt
New Contributor III
  • 1 kudos

@Louis_Frolio Hi Louis,I am using Enterprise databricks version. Could you please share the exact name of the features that needs to be enabled that I can ask my account team to enable this for us ?Thanks

  • 1 kudos
6 More Replies
PedroRodriguezx
by New Contributor
  • 373 Views
  • 1 replies
  • 1 kudos

Databricks RoadMap

Is there any RoadMap delivered by Databricks Team ? I'm doing some tests in my enviroment in order to decide if I will use Databricks as my GenAI platform. However, there are a lot of features in beta stage that can be usefull or not, and to properly...

  • 373 Views
  • 1 replies
  • 1 kudos
Latest Reply
Louis_Frolio
Databricks Employee
  • 1 kudos

Hey Pedro, Databricks does not have a public-facing roadmap.  Cheers, Lou.

  • 1 kudos
FRB1984
by New Contributor II
  • 794 Views
  • 1 replies
  • 0 kudos

Error installing python dependencies

Hi guys! At first, I am sorry if my question is dumb but I am newbie to Databricks! I am just starting and as I try to install ibm_db as a dependency for my python notebook, I get this error :Collecting ibm_db (from -r /tmp/tmp-e8a4538f00b7498fab13c2...

  • 794 Views
  • 1 replies
  • 0 kudos
Latest Reply
SP_6721
Honored Contributor
  • 0 kudos

Hi @FRB1984 ,It looks like the ibm_db package doesn’t support the Python version currently used in your Databricks environment.Instead of relying on ibm_db, you can use Spark’s JDBC connector with the IBM DB2 JDBC driver for a more stable and support...

  • 0 kudos
Khaja_Zaffer
by Contributor III
  • 2119 Views
  • 5 replies
  • 4 kudos

Resolved! community portal needs some updates

Hello  databricks teamcan you please make me understand the search bar on community portal doesnt show results. is it for me or for everyone?Also this portal needs good updates. 

  • 2119 Views
  • 5 replies
  • 4 kudos
Latest Reply
Khaja_Zaffer
Contributor III
  • 4 kudos

@Advika Hey thank you so much.  Just wow. Its working now. I dont know what you did but it works as expected. thank you so much 

  • 4 kudos
4 More Replies
matthiasn
by New Contributor III
  • 698 Views
  • 2 replies
  • 0 kudos

Share notebooks with a AD-user assigned via group

Hi everybody,I try to share a notebook with a user that was assigned to the workspace via a AD-Group (using the new automatic sync). But I only see users directly assigned to the workspace. My expectation would be that I see all users that are within...

  • 698 Views
  • 2 replies
  • 0 kudos
Latest Reply
Louis_Frolio
Databricks Employee
  • 0 kudos

Here are some helpful tips/tricks, and general guidance.   Sharing notebooks with users who are members of Active Directory (AD) groups can sometimes be challenging due to the way Azure Databricks handles user permissions. Here are some important poi...

  • 0 kudos
1 More Replies
nishaath
by New Contributor
  • 3254 Views
  • 1 replies
  • 0 kudos

Workspace creation is getting failed constanly

We are trying to create a workspace with custome VPC. But the creation is getting failed constantly.The error doesn't have any description which makes the troubleshoting hard.Does anyboday had similar issue like this before and has a solution? Thanks...

nishaath_1-1740998508239.png
  • 3254 Views
  • 1 replies
  • 0 kudos
Latest Reply
kamal_ch
Databricks Employee
  • 0 kudos

Hi Nisha, If you are still having the issue, I recommend opening a Support ticket with the workspace ID and any error details so that we can investigate. Regards Kamal

  • 0 kudos
jAAmes_bentley
by Contributor
  • 1111 Views
  • 1 replies
  • 1 kudos

Resolved! Databricks Apps - X-Forwarded-Access-Token not available

Hi all,I've recently been getting started with Databricks Apps, and have been trying to use on-behalf-of access. However, it seems that the X-Forwarded-Access-Token is not coming through in my app headers. I have lots of other X-Forwarded ones such a...

jAAmes_bentley_0-1753219027973.png jAAmes_bentley_1-1753219116820.png
  • 1111 Views
  • 1 replies
  • 1 kudos
Latest Reply
jAAmes_bentley
Contributor
  • 1 kudos

For anybody who comes across this post with the same issue, this issue is caused by enabling the on-behalf-of preview after deploying an App. The solution is to fully stop the App and then start it again after the preview has been enabled. Redeployin...

  • 1 kudos
hmnilp
by New Contributor II
  • 1029 Views
  • 2 replies
  • 0 kudos

Error with file events in external location (via pub/sub)

Hi everyone!I am new here and in Databricks world.I was trying to enable a trigger event on file uploading in GCP bucket in order to start a job following this tutorial: https://docs.databricks.com/gcp/en/connect/unity-catalog/cloud-storage/manage-ex...

hmnilp_0-1752831905771.png hmnilp_1-1752832082230.png hmnilp_3-1752832431023.png
  • 1029 Views
  • 2 replies
  • 0 kudos
Latest Reply
Khaja_Zaffer
Contributor III
  • 0 kudos

Hey @hmnilp did you contact databricks, may be there is a feature to be enabled. 

  • 0 kudos
1 More Replies
RameshRetnasamy
by Contributor
  • 23604 Views
  • 26 replies
  • 26 kudos

Resolved! Unable to login to Azure Databricks Account Console

I have a personal Azure pay-as-you-go subscription in which I have the 'Global Administrator' role. I am also the databricks account administrator.Until two weeks ago, I was able to access the databricks account console without any issues, but I am f...

Screenshot 2024-08-07 at 12.13.53.png Screenshot 2024-08-07 at 12.15.30.png Screenshot 2024-08-07 at 12.17.20.png Screenshot 2024-08-07 at 12.18.35.png
Administration & Architecture
account-console
Databricks
  • 23604 Views
  • 26 replies
  • 26 kudos
Latest Reply
satyaki_guha
New Contributor II
  • 26 kudos

Hi @RameshRetnasamy thanks for your solution.I was able to login to Databricks Account Console but was unable to set the metastore path.Something got broke along the way.It would definitely be beneficial if we could have the same email address for ou...

  • 26 kudos
25 More Replies
ZD
by New Contributor III
  • 6164 Views
  • 8 replies
  • 1 kudos

Resolved! Creating Azure Databricks Workspace Without NAT Gateway

Hello,Recently, when I create a new Databricks workspace on Azure, it automatically create a NAT Gateway which incurs additional cost !When creating the workspace, I don't choose secure cluster connectivity, so I'm expecting not to have a NAT Gateway...

nat 1.jpg nat 2.jpg
  • 6164 Views
  • 8 replies
  • 1 kudos
Latest Reply
ZD
New Contributor III
  • 1 kudos

Make sure to disable the following option when creating your workspace, as it has recently become enabled by default.Deploy Azure Databricks workspace with Secure Cluster Connectivity (No Public IP) 

  • 1 kudos
7 More Replies
Ninad-Mulik
by New Contributor III
  • 972 Views
  • 2 replies
  • 1 kudos

Resolved! How do I access Hive Metastore via API?

Hi,Is there any Databricks REST API to directly access Hive Metastore tables (like list of tables, schemas, columns) without using Unity Catalog or SQL Warehouse?I’m specifically looking to fetch metadata from hive_metastore.Thanks.

  • 972 Views
  • 2 replies
  • 1 kudos
Latest Reply
SP_6721
Honored Contributor
  • 1 kudos

Hi @Ninad-Mulik ,As far as I know, there’s no Databricks REST API to directly access Hive Metastore metadata without using Unity Catalog or SQL Warehouses. The legacy Hive Metastore only supports SQL-based queries. REST APIs for metadata access are o...

  • 1 kudos
1 More Replies
Chinu
by New Contributor III
  • 971 Views
  • 1 replies
  • 1 kudos

Resolved! Guidance on Managing Databricks Apps Whitelisting

Hi,Our workspace users currently have permissions to create Databricks apps, and we’ve observed a rise in associated costs. To address this, we’re developing an administrative notebook to:Check whether an app is whitelisted/approved from the admins.A...

  • 971 Views
  • 1 replies
  • 1 kudos
Latest Reply
Louis_Frolio
Databricks Employee
  • 1 kudos

Here are some helpful tips that you might find useful:   Summary of Best Practices and Recommendations for App Whitelisting and Automated App Removal in Databricks 1. Overview and Whitelisting Strategy To control costs and maintain governance over Da...

  • 1 kudos
Kim3
by New Contributor II
  • 2618 Views
  • 1 replies
  • 0 kudos

Delete Azure Databricks Workspace resource but reference remains in Account Console

As the title says. I have deleted both the Azure Databricks Workspace resource and the resource group it was located in. However, I can still see the workspace in the account console. Also, it appears in the response from GET https://accounts.azureda...

  • 2618 Views
  • 1 replies
  • 0 kudos
Latest Reply
Jcpenuela
New Contributor II
  • 0 kudos

Hi Kim3, same problem here, did you find a solution for this?Thanks!

  • 0 kudos
rathorer
by New Contributor III
  • 2086 Views
  • 0 replies
  • 2 kudos

Building MultiTenant Architecture on Databricks Platform

This use case demonstrates how a SaaS product can be deployed for multiple customers or business units, ensuring data isolation at every layer through workspace separation, fine-grained access control with Unity Catalog, and secure processing using U...

rathorer_0-1753157201307.png rathorer_1-1753157291951.png rathorer_2-1753157577943.png rathorer_3-1753157853253.png
Administration & Architecture
Architecture
multi-tenant
Security framework
  • 2086 Views
  • 0 replies
  • 2 kudos