- 200 Views
- 2 replies
- 1 kudos
Databricks Apps: AWS Secret Manager Access
We are exploring Databricks Apps.We want Databricks APP to interact with AWS Secret Manager. How we can configure this and configure IAM on AWS side for this to take place.@app
- 200 Views
- 2 replies
- 1 kudos
- 1 kudos
Thanks @Alberto_Umana .. Yes we will try to use databricks secret, that can be helpful. Couple of other questions on Databricks App 1) Can we use Framework other than mentioned in documentation( Streamlit,Flask,Dash,Gradio,Shiny).2) If required can w...
- 1 kudos
- 165 Views
- 4 replies
- 0 kudos
Different NCC having same subnets
Hello,We are forwarding this Microsoft tutorial to secure our storage access:https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serverless-firewallWe have a weird behavior when we create several NCCs in th...
- 165 Views
- 4 replies
- 0 kudos
- 0 kudos
Ok, so no, I correctly set the subnets of my NCC in the Virtual Networks setting as documented:https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serverless-firewallThis setting is working fine, without th...
- 0 kudos
- 112 Views
- 1 replies
- 2 kudos
How can you enable {{"data_security_mode": "USER_ISOLATION" }} sql warehouses
Hi,I want to create an sql warehouse with {{"data_security_mode": "USER_ISOLATION" }} however, I dont find the section to get the json file of my cluster. Thanks
- 112 Views
- 1 replies
- 2 kudos
- 2 kudos
Hello @carlos_tasayco, Access mode are not configurable in SQL warehouses. Please see: https://docs.databricks.com/api/workspace/warehouses/create
- 2 kudos
- 71 Views
- 0 replies
- 1 kudos
Get managedResourceGroup from serverless
Hello,In my job I have a task where I should modify a notebook to get dynamically the environment, for example:This is how we get it:dic = {"D":"dev", "Q":"qa", "P":"prod"}managedResourceGroup = spark.conf.get("spark.databricks.xxxxx")xxxxx_Index = m...
- 71 Views
- 0 replies
- 1 kudos
- 114 Views
- 1 replies
- 0 kudos
Resolved! How can i run a single task in job from Rest API
How can I run a single task in a job that has many tasks?I can do it in the UI, but I can’t find a way to do it using the REST API. Does anyone know how to accomplish this?
- 114 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @200842 , Greetings! It is currently not possible to run a single task or a group of tasks from a job without running the entire job from the API. Leave a like if this helps, followups are appreciated.Kudos, Ayushi
- 0 kudos
- 196 Views
- 3 replies
- 3 kudos
Subscription and billing: Directly or via AWS?
Is there any difference in price & features if I subscribe to DataBricks directly or via AWS account?Any startup discount programs available?
- 196 Views
- 3 replies
- 3 kudos
- 3 kudos
No problem, and there is no difference in the pricing between subscribing to Databricks directly or via the AWS Marketplace, it's just about billing mechanism.
- 3 kudos
- 270 Views
- 5 replies
- 1 kudos
Resolved! Payment receipts of Databricks payments
Hello experts,I am trying to get receipts for the monthly payments done to Databricks. I need them for the financial department of the organization I am working for. The only billing information I get access to is the usage dashboards and the tables ...
- 270 Views
- 5 replies
- 1 kudos
- 272 Views
- 3 replies
- 2 kudos
Resolved! DLT-Asset bundle : Pipelines do not support a setting a run_as user that is different from the owner
Hello !We're using Databricks asset bundles to deploy to several environments using a devops pipeline. The service principal running the CICD pipeline and creating the job (owner) is not the same as the SPN that will be running the jobs (run_as).This...
- 272 Views
- 3 replies
- 2 kudos
- 2 kudos
By default who creates pipeline is the owner/run_as. However, yes this is a feature in progress. I wouls still encorage you to share your request her https://docs.databricks.com/en/resources/ideas.html#ideas ...
- 2 kudos
- 202 Views
- 3 replies
- 0 kudos
Resolved! Issue in building databricks source code on ARM platform
Hi,I am attempting to build the Databricks Koalas source code on an ARM platform but am encountering an error. Could anyone provide specific steps for building it on ARM?
- 202 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @mohakgoel, You’re on the right track with the steps you’ve mentioned for building Koalas. It’s important to note that building Koalas is not the same as building the entire Databricks source code. Koalas is an open-source project that provides a ...
- 0 kudos
- 266 Views
- 5 replies
- 0 kudos
Resolved! Using Databricks Docker CLI image with Asset Bundles - Azure
I'm trying to deploy asset bundle from a CI/CD pipeline, I'd like to use the docker databricks CLI image for that, but I can't get it to authenticate. I'm using entra service principal for my deployments and we are using TeamCity as our CI/CD tool. T...
- 266 Views
- 5 replies
- 0 kudos
- 0 kudos
OK, tested and now everything is working - according to the docs bundle settings are of highest priority, https://learn.microsoft.com/en-us/azure/databricks/dev-tools/cli/authentication#auth-evalNo wonder none of my env vars or .databrickscfg setting...
- 0 kudos
- 1150 Views
- 1 replies
- 0 kudos
We are getting AttributeError: module 'numpy.typing' has no attribute 'NDArray' with tidal jobs
File "/databricks/python/lib/python3.8/site-packages/PIL/_typing.py", line 10, in <module>NumpyArray = npt.NDArray[Any]AttributeError: module 'numpy.typing' has no attribute 'NDArray' we are using 10.4 runtime version , please suggest
- 1150 Views
- 1 replies
- 0 kudos
- 0 kudos
The file path of the above error is suggesting it is from the Pillow library dependency: "/databricks/python/lib/python3.8/site-packages/PIL/_typing.py" Pillow's _typing.py checks for NumPy >=1.21 while 10.4 runtime has these versions installed ...
- 0 kudos
- 796 Views
- 1 replies
- 1 kudos
Troubleshooting Cluster
We had a failure on a previously running fact table load (our biggest one) and it looked like an executor was failing due to a timeout error. As a test we upped the cluster size and changed the spark.executor.heartbeatinterval to 300s and the spark....
- 796 Views
- 1 replies
- 1 kudos
- 1 kudos
The XXKDA error code is a general indicator for task scheduler issues or SPARK_JOB_CANCELLED.
- 1 kudos
- 136 Views
- 1 replies
- 0 kudos
Community Accounts Getting Deleted Automatically
This is the second time in the same week, the accounts are getting vanished without any prior information. All the saved work has no backup I guess and no one would take responsibility because it a community version.Not a good thing to happen twice, ...
- 136 Views
- 1 replies
- 0 kudos
- 0 kudos
I would suggest to follow steps in https://community.databricks.com/t5/support-faqs/databricks-community-sso-august-3rd-2024/ta-p/78459 , if issue persist please contact databricks-community@databricks.com
- 0 kudos
- 189 Views
- 1 replies
- 0 kudos
Account missing
I have been using my community for the past two weeks. Now, while trying to log in, it shows this: Can anyone help me fix this? Thanks in advance!
- 189 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello, @udhayakumar!You can find helpful resources for Databricks Community Edition here. If the available resource doesn’t resolve your concern, please submit a ticket with Databricks Support team for further assistance.
- 0 kudos
- 334 Views
- 3 replies
- 3 kudos
Drop table - permission management
Hello,I'm trying to wrap my head around the permission management for dropping tables in UC enabled schemas.According to docs: To drop a table you must have the MANAGE privilege on the table, be its owner, or the owner of the schema, catalog, or meta...
- 334 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @PiotrM, I see there is a feature request already in place. It's been considered for the future: https://databricks.aha.io/ideas/ideas/DB-I-7480
- 3 kudos
