- 227 Views
- 0 replies
- 0 kudos
Custom Runtime marketplace
Hi! Is there a possibility to share the solution accelerator on the custom runtime via the databricks marketplace?
- 227 Views
- 0 replies
- 0 kudos
Forum Posts
- 366 Views
- 2 replies
- 1 kudos
Resolved! Delta sharing gold layer data within organisation but outside of Vnet
Hi all,The organisation I’m working for has a data engineering team using medallion architecture and wants to share materialised views in the gold layer to members of the organisation who do not have access to the Vnet to be delivered in power BI rep...
- 366 Views
- 2 replies
- 1 kudos
Latest Reply
- 942 Views
- 1 replies
- 1 kudos
session expired Error
Within two minutes of logging into Databricks Community Edition, I am receiving the following error message: "Session expired." The session has ended; please log in again." I've attempted to log in again, but the message "Authenticate the session is ...
- 942 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @Rajatkumar002 ,You can check now. I think it's been resovled now.
- 1 kudos
- 337 Views
- 1 replies
- 0 kudos
Terraform databricks new feature
I recently saw a new feature in databricks that allows to set an upper limit of run time for jobs, so it fails when the limit is breached. See image below:Are we able to terraform code this new feature?
- 337 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Isn't that just timeout_seconds?That UI element looks like it corresponds to that property in the REST API (https://docs.databricks.com/api/azure/workspace/jobs/create#timeout_seconds), and that same top-level argument in Terraform, https://registry....
- 0 kudos
- 330 Views
- 1 replies
- 0 kudos
Account Access token
Can you please help me with how to authenticate account level rest apis?curl --request GET https://accounts.cloud.databricks.com/api/2.0/accounts/{account_id}/usageI use Permanent Access Token to authenticate workspace APIs but it's not working for w...
- 330 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @shubhdhanuka ,To get access to account API you cannot use personal access token. Instead, you must use the Microsoft Entra ID tokens of Azure Databricks account admins. Azure Databricks account admins can be users or service principals.Below you ...
- 0 kudos
- 444 Views
- 1 replies
- 1 kudos
Databricks + Apache Iceberg = advantageous or wasted effort due to duplicate functionality ?
Trying to design a Lakehouse. Spark is at the base layer. Now wondering if adding Apache Iceberg sitting below Spark will be of help, or, not ? Preferring Iceberg for its auto indexing, ACID query facilities over big hetergenous datasets. Wonder if i...
- 444 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
Hello, if you're planning on building your own open source stack of spark+iceberg, it can be a good choice. If you're on Databricks, however, you're going to miss out a *lot* on delta features that are baked into the platform. Specifically compute +...
- 1 kudos
- 525 Views
- 1 replies
- 0 kudos
Resolved! Prevent service principal UUID from appearing on job name
Hello!I am using service principal id to authenticate my databricks bundle. But when the job runs, this id is automatically appended to both the name and tags column on the jobs run page. In my databricks.yml file I have name: "[${var.environment}]" ...
- 525 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi!Sounds like the "development" mode. DAB will automatically prefix your jobname with <env> <user name> if you set "mode" to "development" in the databricks.yml file. The name lookup for service principals apparently doesn't work nicely and you get ...
- 0 kudos
- 2181 Views
- 2 replies
- 3 kudos
Unable to read resources - Unsupported Protocol Scheme (Terraform AWS)
Hello everyone!Over the last few weeks my company has been trying to deploy a Databricks workspace on AWS adapted to the customer's needs, using Terraform. To do this, we started from a base code on Databricks own github (https://github.com/databrick...
- 2181 Views
- 2 replies
- 3 kudos
Latest Reply
- 338 Views
- 1 replies
- 2 kudos
List deployed Databricks asset bundles (DABs)?
Is there a databricks cli command or REST API to list all the DABs that have been deployed to a workspace?
- 338 Views
- 1 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi @Schofield ,Unfortunately, I don't think there is out of the box command that will provide you this information yet. As a workaround, you can try write some code that will extract this information from REST API. For example, you can use /api/2.1/j...
- 2 kudos
- 422 Views
- 2 replies
- 0 kudos
Disable Vector Search
Hello all,Is there a way to disable Vector Search? It's not vetted in my organization and I don't want the users to be able to create it.Best,Pawel
- 422 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Pawel_Pilkowski , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your...
- 0 kudos
- 872 Views
- 2 replies
- 1 kudos
Databricks platform setup best practices in Azure
Could you please help me on the prerequisites, configuring steps and best practices for the data bricks platform and workspace setup in Azure(include subscription, subnet /network configuration ,storage configuration best practices etc).
- 872 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @Phani1 , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback...
- 1 kudos
- 1339 Views
- 2 replies
- 0 kudos
Resolved! Databricks Spot Instance: Completion Guarantee
Databricks allows to use spot instances for worker nodes. I consider to use them for interactive clusters. Do I have a gurantee that code will be completed without any errors even if spot instances are evicted? I would accept execution delays but no ...
- 1339 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
You could explore their "SPOT_WITH_FALLBAK" feature. If you don't want your jobs to fail because of eviction but this currently is not supported with interactive clusters. Hoping that they may extend this to all compute options soonCreate a pipeline ...
- 0 kudos
- 574 Views
- 0 replies
- 0 kudos
Databricks (GCP) Cluster not resolving Hostname into IP address
we have #mongodb hosts that must be resolved to private internal loadbalancer ips ( of another cluster ), and that we are unable to add host aliases in the Databricks GKE cluster in order for the spark to be able to connect to a mongodb and resolve t...
- 574 Views
- 0 replies
- 0 kudos
- 477 Views
- 0 replies
- 0 kudos
Databricks AWS ACLs requirement
I'm going through the documentation in databricks for using existing VPC requirements. For the section that covers ACLs, what's the need to have this inbound rule: " ALLOW ALL from Source 0.0.0.0/0. This rule must be prioritized."https://docs.databri...
- 477 Views
- 0 replies
- 0 kudos
- 1307 Views
- 3 replies
- 0 kudos
Unity Catalog read issue
Hello,Our company is POCing the Unity Catalog with Azure as provider.We have 2 subscriptions that contains 1 databricks each and 1 ADLS GEN2 each.Initially we have the default `hive_metastore` connected to the ADLS GEN2. I've created a secret scope a...
- 1307 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello,The UC catalog container is sitting in Sub B.Basically we have a Spoke and Hub configuration for each Subscriptions. Each Subscriptions can access to any ressource inside it own subscription with some PE. But to access other Subscription ressou...
- 0 kudos