- 764 Views
- 2 replies
- 3 kudos
Resolved! Best practice:Using Databricks managed storage vs customer‑owned ADLS for enterprise production data
We are currently setting up Azure Databricks for enterprise analytics and wanted to validate our storage architecture against Databricks best practices.Today, we are ingesting data directly from external enterprise sources (Oracle DB, SQL Server, etc...
- 764 Views
- 2 replies
- 3 kudos
- 3588 Views
- 1 replies
- 0 kudos
Lakehouse Federation - Unable to connect to Snowflake using "PEM Private Key"
Hi,I'm currently using Lakehouse Federation feature on databricks to run queries against Snowflake datawarehouse. Today I'm using a service credential to establish the connection (user id & pwd), but I have to change it to use private key. I tried us...
- 3588 Views
- 1 replies
- 0 kudos
- 0 kudos
Can you check if you have encrypted your keys . The Snowflake JDBC driver does not support authentication with encrypted private keys. https://docs.databricks.com/aws/en/query-federation/snowflake-pem#limitationsTry generating the key with no encryp...
- 0 kudos
- 349 Views
- 1 replies
- 1 kudos
Resolved! Downgrade from Enterprise to Premium Plan
We upgraded to Enterprise to test something and we want now to switch back to Premium Plan. How can I achieve this. Am I forced to create a new account and delete this one?
- 349 Views
- 1 replies
- 1 kudos
- 1 kudos
No need to cancel your account. There is no self-service option to downgrade your account. Please submit a support ticket and work with your Databricks account team to downgrade.
- 1 kudos
- 1130 Views
- 4 replies
- 0 kudos
Databricks Default Catalog with House Icon
Hi everyone,I have a question regarding workspace catalogs in Databricks with Unity Catalog.In our setup, when a new workspace is created and automatically assigned to a Unity Catalog metastore (with automatic workspace catalog creation enabled), a c...
- 1130 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @vziog, I can see the earlier suggestion about using the kebab menu Rename didn't fully solve the problem. There are actually a few distinct concepts at play here, and understanding the difference is the key. WHAT IS THE WORKSPACE CATALOG (HOUSE I...
- 0 kudos
- 2643 Views
- 6 replies
- 0 kudos
Resolved! Any documentation mentioning connectivity from Azure SQL database connectivity to Azure Databricks
Any documentation available to connect from the Azure SQL database to Azure Databricks SQL workspace. We created a SQL warehouse personal access token for a user in a different team who can connect from his on-prem SQL DB to Databricks using the conn...
- 2643 Views
- 6 replies
- 0 kudos
- 482 Views
- 2 replies
- 1 kudos
Resolved! Any recommended way for a different app to start their dependent job based on Databricks job?
How can we configure a job in a different Azure application to be triggered after the completion of an Azure Databricks job? Once the Databricks job is successful, the job in the third-party application hosted in Azure should start. I attempted to us...
- 482 Views
- 2 replies
- 1 kudos
- 1 kudos
Thank you. Thank you for the detailed answer!I have tested the Azure function way and also using an Azure runbook as well. Both works fine.Also tested the option of adding as the final task and a condition to "if all other notebooks" successful, the...
- 1 kudos
- 21791 Views
- 5 replies
- 1 kudos
Resolved! databricks OAuth is not supported for this host
I'm trying to deploy using Databricks Asset Bundles via an Azure DevOps pipeline. I keep getting this error when trying to use oauth:Error: default auth: oauth-m2m: oidc: databricks OAuth is not supported for this host. Config: host=https://<workspac...
- 21791 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi @bradleyjamrozik, thank you for posting your question. You will need to use ARM_ variables to make it work Specifically ARM_CLIENT_ID ARM_TENANT_ID ARM_CLIENT_SECRET https://learn.microsoft.com/en-us/azure/databricks/dev-tools/auth#environment-3 f...
- 1 kudos
- 1950 Views
- 19 replies
- 13 kudos
Resolved! Databricks kicking me out every 2 minutes
After the session expire every 2 minutes as show below, I can't log in afterwards as shown further below. What the heck is this? Then I have to wait the platform own will to allow me to log in many minutes later.
- 1950 Views
- 19 replies
- 13 kudos
- 13 kudos
I can definitely tell it's improved—it's been 30 minutes and it hasn't logged me out yet.Thank you so much!
- 13 kudos
- 3476 Views
- 15 replies
- 2 kudos
Need Help - System tables that contains all databricks users, service principal details !!
Hi all - I am trying to create a dashabord where I need to list down all users and service principals along with groups and understand their databricks usages. Is there any table available in Databricks that contains user, service principal details. ...
- 3476 Views
- 15 replies
- 2 kudos
- 516 Views
- 1 replies
- 1 kudos
Custom MCP for Genie via Apps
I wanted to create a custom MCP to help Genie generate some code - its a simple API callI had everything set up - the app, the secrets, everything was deployedBut Genie won't connect, and when I try to add "On Behalf of User Authorization" roles it d...
- 516 Views
- 1 replies
- 1 kudos
- 1 kudos
Check these: First, make sure you're adding the authorization roles through the App's settings in the workspace where Genie will actually use it, not just where you developed it. The roles need to be set in the target workspace. Second, try refreshin...
- 1 kudos
- 369 Views
- 1 replies
- 1 kudos
OJDBC8 stoped working on LTS 17.3
Hi, we upgraded our clusters to LTS 17.3 as we needed some functionality from that release and our OJDBC8 drivers have stopped working. Downgraded back to LTS 14 and its all fine again.What are the latest OJDBC drivers that are compatible with LTS 1...
- 369 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @ScottDunk, Looking at the release notes for LTS 14 and LTS 17.3, it appears to be a compatibility issue... LTS 14:LTS 17.3: As you can see from the images above, Databricks 17.3 LTS upgrades the cluster JVM to Java 17. Oracle’s own compatibility ...
- 1 kudos
- 387 Views
- 1 replies
- 1 kudos
Resolved! DLT pipeline production deployment with AWS
Hi,We are currently working with pyspark where we are doing the ETL as well as data quality check. We are converting our code in to the wheel package to go in to the prod and manage the version for better stability.We are running Databricks with AWS...
- 387 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @satycse06, Have you considered Declarative Automation Bundles (Previously called Databricks Asset bunddles) for this? This is exactly the type of problem it solves. You can still keep your DLT code and a databricks.yml bundle file in Azure DevOps...
- 1 kudos
- 1313 Views
- 4 replies
- 3 kudos
Resolved! Security & Compliance understanding on LLM Usage in Databricks Genie and Agentbricks
Hi everyone,With the increasing focus on security and compliance for AI Agents and LLMs, I wanted to get some clarity on a couple of points related to Databricks Genie and Agentbricks.Could someone help provide detailed information on the following, ...
- 1313 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @abhijit007, Please take a look at these pages. They answer your queries in detail for Genie. https://docs.databricks.com/genie - Covers architecture and how it works. Also covers security. https://docs.databricks.com/databricks-ai/databricks-ai-...
- 3 kudos
- 474 Views
- 2 replies
- 1 kudos
Resolved! Newly added workspace users do not appear immediately in WorkspaceClient().users.list() or SCIM API
Hello,I’m trying to retrieve the list of users in a Databricks workspace.I am currently using both the Databricks SDK and the SCIM API: from databricks.sdk import WorkspaceClient w = WorkspaceClient() users = list(w.users.list())and also: import re...
- 474 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @discuss_darende, I agree with Pradeep here. In practice, there can be a delay before the identity and its memberships are fully visible everywhere, especially if you’re on Azure and using AIM or a SCIM connector from your IdP. The delay isn’t do...
- 1 kudos
- 1008 Views
- 1 replies
- 0 kudos
Issues when configuring keystore spark config for pyspark to mongo atlas X.509 connectivity
Step followed - Step1: To add init script that will copy the keystore file in the tmp location.Step2: To add spark config in cluster advance options - spark.driver.extraJavaOptions -Djavax.net.ssl.keyStore=/tmp/keystore.jks -Djavax.net.ssl.keyStorePa...
- 1008 Views
- 1 replies
- 0 kudos
- 0 kudos
The error isn’t actually about MongoDB. HikariCP failing on port 3306 via DataNucleus is your Hive Metastore losing its SSL connection to MySQL on the driver.Setting javax.net.ssl.keyStore globally in extraJavaOptions overwrites the default JVM trust...
- 0 kudos
