- 721 Views
- 1 replies
- 0 kudos
De facto Standard for Databricks on AWS
Hello,I am working on creating an architecture diagram for Databricks on AWS.I would like to adopt the de facto standard used by enterprises. Based on my research, I have identified the following components:Network: Customer-managed VPC,Secure Cluste...
- 721 Views
- 1 replies
- 0 kudos
- 0 kudos
I would not call it a 'standard' but a possible architecture. The great thing about the cloud is you can complete the puzzle in many ways and make it as complex or as easy as possible.Also I would not consider Fivetran to be standard in companies. ...
- 0 kudos
- 1213 Views
- 6 replies
- 0 kudos
permissions tab is missing from policy UI
Hi Team.When I try to create a new policy the permissions tab is missing.I am an account admin.Any ideas why?Many thanks.Dave.
- 1213 Views
- 6 replies
- 0 kudos
- 0 kudos
@bigger_dave If you are trying to create a compute policy, permissions tab should be available during configuration. If you wanted to grant to an existing policy, then permissions tab is available once you choose edit the policy. If you are looking f...
- 0 kudos
- 1546 Views
- 6 replies
- 0 kudos
Resolved! Updating Workspace Cluster
Hello,My organization is experiencing difficulties updating our Google Kubernetes Engine (GKE) cluster.We've reviewed the official GKE documentation for automated cluster updates, but it appears to primarily focus on AWS integrations. We haven't foun...
- 1546 Views
- 6 replies
- 0 kudos
- 7009 Views
- 5 replies
- 2 kudos
Resolved! How to enable Genie?
Hi All,Based on the article below to enable Genie one needs to:1. Enable Azure AI services-powered featuresThat is done:2. Enable Genie must be enabled from the Previews pageI do not see Genie among Previews:I am using Azure Databricks. Any idea how ...
- 7009 Views
- 5 replies
- 2 kudos
- 2 kudos
I can access to preview on account level but can't see Genie in Previews
- 2 kudos
- 4056 Views
- 2 replies
- 3 kudos
Databricks shared workspace
We have a Self service portal through which users can launch databricks clusters of different configurations. This portal is set up to work in Dev, Sandbox and Prod environments. We have configured databricks workspaces only for Sandbox and Prod por...
- 4056 Views
- 2 replies
- 3 kudos
- 3 kudos
@Alberto_Umana Thanks for sharing doc linksWe have exact same set up to support shared databricks workspace. But still Im facing issue while adding instance profileI am trying to add AWS Instance Profile created in source AWS Account (No databricks w...
- 3 kudos
- 559 Views
- 1 replies
- 2 kudos
displayHTML <a href="#id"> not working
Many packages output a html-report, e.g. ydata-profiler. The report contains links to other parts of the report. But when the user clicks the links a new window is opened instead of scrolling to the correct section of the displayed html.Could this be...
- 559 Views
- 1 replies
- 2 kudos
- 2 kudos
Hello @invalidargument, Currently, there is no direct support from the Databricks end to modify this behavior without using such a workaround. The displayHTML function in Databricks renders HTML content within an iframe, and the injected JavaScript h...
- 2 kudos
- 1155 Views
- 1 replies
- 2 kudos
Resolved! Databricks Docker CLI image - how to debug Terraform when deploying Asset Bundle
We're having issues when deploying asset bundle using docker databricks cli image.Validation part passes OK:14:21:15 Name: test 14:21:15 Target: prototype-dev 14:21:15 Workspace: 14:21:15 Host: https://adb-xxx.azuredatabricks.net/ 14:21:15 ...
- 1155 Views
- 1 replies
- 2 kudos
- 2 kudos
OK, as it turns out - in order to bypass proxy we needed to set no_proxy env variable in both upper and lower case (!), like this:NO_PROXY="adb-xxx.azuredatabricks.net" docker run \ -v %teamcity.build.checkoutDir%:/my-bundle \ -v %teamcity.build...
- 2 kudos
- 1070 Views
- 2 replies
- 1 kudos
Databricks Apps: AWS Secret Manager Access
We are exploring Databricks Apps.We want Databricks APP to interact with AWS Secret Manager. How we can configure this and configure IAM on AWS side for this to take place.@app
- 1070 Views
- 2 replies
- 1 kudos
- 1 kudos
Thanks @Alberto_Umana .. Yes we will try to use databricks secret, that can be helpful. Couple of other questions on Databricks App 1) Can we use Framework other than mentioned in documentation( Streamlit,Flask,Dash,Gradio,Shiny).2) If required can w...
- 1 kudos
- 2596 Views
- 4 replies
- 0 kudos
Different NCC having same subnets
Hello,We are forwarding this Microsoft tutorial to secure our storage access:https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serverless-firewallWe have a weird behavior when we create several NCCs in th...
- 2596 Views
- 4 replies
- 0 kudos
- 0 kudos
Ok, so no, I correctly set the subnets of my NCC in the Virtual Networks setting as documented:https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serverless-firewallThis setting is working fine, without th...
- 0 kudos
- 749 Views
- 1 replies
- 2 kudos
How can you enable {{"data_security_mode": "USER_ISOLATION" }} sql warehouses
Hi,I want to create an sql warehouse with {{"data_security_mode": "USER_ISOLATION" }} however, I dont find the section to get the json file of my cluster. Thanks
- 749 Views
- 1 replies
- 2 kudos
- 2 kudos
Hello @carlos_tasayco, Access mode are not configurable in SQL warehouses. Please see: https://docs.databricks.com/api/workspace/warehouses/create
- 2 kudos
- 1334 Views
- 3 replies
- 3 kudos
Subscription and billing: Directly or via AWS?
Is there any difference in price & features if I subscribe to DataBricks directly or via AWS account?Any startup discount programs available?
- 1334 Views
- 3 replies
- 3 kudos
- 3 kudos
No problem, and there is no difference in the pricing between subscribing to Databricks directly or via the AWS Marketplace, it's just about billing mechanism.
- 3 kudos
- 2323 Views
- 3 replies
- 4 kudos
Resolved! DLT-Asset bundle : Pipelines do not support a setting a run_as user that is different from the owner
Hello !We're using Databricks asset bundles to deploy to several environments using a devops pipeline. The service principal running the CICD pipeline and creating the job (owner) is not the same as the SPN that will be running the jobs (run_as).This...
- 2323 Views
- 3 replies
- 4 kudos
- 4 kudos
By default who creates pipeline is the owner/run_as. However, yes this is a feature in progress. I wouls still encorage you to share your request her https://docs.databricks.com/en/resources/ideas.html#ideas ...
- 4 kudos
- 943 Views
- 3 replies
- 0 kudos
Resolved! Issue in building databricks source code on ARM platform
Hi,I am attempting to build the Databricks Koalas source code on an ARM platform but am encountering an error. Could anyone provide specific steps for building it on ARM?
- 943 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @mohakgoel, You’re on the right track with the steps you’ve mentioned for building Koalas. It’s important to note that building Koalas is not the same as building the entire Databricks source code. Koalas is an open-source project that provides a ...
- 0 kudos
- 2858 Views
- 5 replies
- 0 kudos
Resolved! Using Databricks Docker CLI image with Asset Bundles - Azure
I'm trying to deploy asset bundle from a CI/CD pipeline, I'd like to use the docker databricks CLI image for that, but I can't get it to authenticate. I'm using entra service principal for my deployments and we are using TeamCity as our CI/CD tool. T...
- 2858 Views
- 5 replies
- 0 kudos
- 0 kudos
OK, tested and now everything is working - according to the docs bundle settings are of highest priority, https://learn.microsoft.com/en-us/azure/databricks/dev-tools/cli/authentication#auth-evalNo wonder none of my env vars or .databrickscfg setting...
- 0 kudos
- 5240 Views
- 1 replies
- 0 kudos
We are getting AttributeError: module 'numpy.typing' has no attribute 'NDArray' with tidal jobs
File "/databricks/python/lib/python3.8/site-packages/PIL/_typing.py", line 10, in <module>NumpyArray = npt.NDArray[Any]AttributeError: module 'numpy.typing' has no attribute 'NDArray' we are using 10.4 runtime version , please suggest
- 5240 Views
- 1 replies
- 0 kudos
- 0 kudos
The file path of the above error is suggesting it is from the Pillow library dependency: "/databricks/python/lib/python3.8/site-packages/PIL/_typing.py" Pillow's _typing.py checks for NumPy >=1.21 while 10.4 runtime has these versions installed ...
- 0 kudos
