- 2185 Views
- 9 replies
- 0 kudos
Who's hiring? Latest Job Postings from the Databricks Community!
More than 10,000 organizations worldwide — including Block, Comcast, Condé Nast, Rivian, Shell and over 60% of the Fortune 500 — rely on the Databricks Data Intelligence Platform to take control of their data and put it to work with AI. Use this thre...
- 2185 Views
- 9 replies
- 0 kudos
- 0 kudos
Cedar is seeking a seasoned Engineering Director to lead our Data Integration and Platforms team. This role will help us tackle challenges of scale, quality and insight as the company grows its market share and product scope. Job Title: Engineering...
- 0 kudos
- 679 Views
- 4 replies
- 0 kudos
Automate run as workflow parameter to default to current user
I am trying to run a workflow within Databricks. I have 2 workflows, workflow one which always runs as the service principal, as all data gets accessed and wrangled within this workflow, and workflow 2 which always defaults to the last run account. I...
- 679 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi, how are you expecting to achieve this? Do you want users who are manually triggering this workflow first update to their run_as? or you want to make this happen programatically?
- 0 kudos
- 475 Views
- 2 replies
- 0 kudos
Create csv and upload on azure
Can some write a sql query , which queries a table like select * from stages.benefit , creates a csv and upload on azure
- 475 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @subhadeep ,You can achieve this in SQL similarly to how you write a dataframe into a table or blob path. We will create an external table pointing to the blob path or mounted blob path. Note that this table does not support ACID transactions and ...
- 0 kudos
- 1108 Views
- 10 replies
- 2 kudos
01_demo_setup error
HelloI was following "Demo: Creating and Working with a Delta Table"while I have a community edition user.The first command in the Notebook is: %run ./setup/01_demo_setup But I got the following error:Notebook not found: Users/<my-email-was-here..>/s...
- 1108 Views
- 10 replies
- 2 kudos
- 2 kudos
Hey!Sad news guys... if you go to Course Logistics Review you can read:"We are pleased to offer a version of this course that also contains hands-on practice via a Databricks Academy Labs subscription. With a Databricks Academy Labs subscription, you...
- 2 kudos
- 441 Views
- 1 replies
- 0 kudos
Databricks app giving 'upstream request timeout '
Hello all,We are developing an app which is based on flask, which is used to download logs from databricks dbfs location. For this useful case we are using databricks inbuilt App feature to deploy our app.While we pass a smaller file it is getting do...
- 441 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey!It looks like the issue you’re facing might be related to the proxy timeout when downloading large files from DBFS. Since modifying the proxy settings might not be an option, there are a couple of alternative approaches you could consider to miti...
- 0 kudos
- 4405 Views
- 3 replies
- 0 kudos
50%-off Databricks certification voucher
Hello Databricks Community Team, I am reaching out to inquire about the Databricks certification voucher promotion for completing the Databricks Learning Festival (Virtual) courses.I completed one of the Databricks Learning Festival courses July 2024...
- 4405 Views
- 3 replies
- 0 kudos
- 521 Views
- 1 replies
- 0 kudos
How to Create Azure Key Vault and Assign Key Vault Administrator Role Using Terraform
Hi all,I’m currently working with Terraform to set up Azure resources, including OpenAI services, and I’d like to extend my configuration to create an Azure Key Vault. Specifically, I want to:Create an Azure Key Vault to store secrets/keys.Assign the...
- 521 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @naveen0142 , 1. Create the Key Vault resource "azurerm_key_vault" "example" { name = var.key_vault_name location = azurerm_resource_group.example.location resource_group_name = azurerm_resource_group.example....
- 0 kudos
- 267 Views
- 0 replies
- 0 kudos
Atualização Incremental e/ou Modelos Compostos (Databricks x Power BI)
Gostaria de deixar meu modelo mais performático no Power BI, mas tenho encontrado algumas dificuldades ao conectá-lo em uma fonte no DataBricks. Queria saber se é possÃvel fazer atualização incremenal e/ou trabalhar com modelos compostos (Direct Quer...
- 267 Views
- 0 replies
- 0 kudos
- 568 Views
- 1 replies
- 0 kudos
User Unable to Access Key Vault Secrets Despite Role Assignment in Terraform
Hi All,I'm encountering an issue where a user is unable to access secrets in an Azure Key Vault, even though the user has been assigned the necessary roles using Terraform. Specifically, the user gets the following error when trying to access the sec...
- 568 Views
- 1 replies
- 0 kudos
- 0 kudos
Are they accessing the Key Vault directly and not through Databricks? If so, based on your Terraform code, they should be able to directly read Secrets in the Azure Key Vault. You've configured the Key Vault with RBAC Authorization and assigned Key ...
- 0 kudos
- 234 Views
- 1 replies
- 0 kudos
Prevent users from running shell commands
Hi, is there any way to prevent users from running shell commands in Databricks notebooks? for example, "%%bash" I read that REVOKE EXECUTE ON SHELL command can be used. but i am unable to make it to work. Thanks in advance for any help.
- 234 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @bvraravind, You can use this spark setting "spark_conf.spark.databricks.repl.allowedLanguages": { "type": "fixed", "value": "python,sql" Over a cluster policy to prevent access to shell commands. https://docs.databricks.com/en/archive/compute/c...
- 0 kudos
- 635 Views
- 2 replies
- 0 kudos
Terraform: Add Key Vault Administrator Role Assignment and Save Outputs to JSON Dynamically in Azure
Hi everyone,I am using Terraform to provision an OpenAI service and its modules along with a Key Vault in Azure. While the OpenAI service setup works as expected, I am facing two challenges:Role Assignment for Key VaultI need to assign the Key Vault ...
- 635 Views
- 2 replies
- 0 kudos
- 0 kudos
For question two, you can use the local_file resource in Terraform: output "openai_api_type" {value = module.openai.api_type} output "openai_api_base" {value = module.openai.api_base} output "openai_api_version" {value = module.openai.api_version} ou...
- 0 kudos
- 3320 Views
- 4 replies
- 1 kudos
Cluster policy type
Hi guys,I am creating a cluster policy through json. "runtime_engine": {"type": "fixed""value": "PHOTON"}When I run the above code... PHOTON option is getting enabled but graying out... What would I specify in type field so that the photon option sho...
- 3320 Views
- 4 replies
- 1 kudos
- 1 kudos
Hey, If you change the type to "allowlist" and provide "PHOTON" and "STANDARD" as options that should fix your issue.here is an example: "runtime_engine": {"type": "allowlist""value": ["PHOTON", "STANDARD"]"defaultValue": "PHOTON"}
- 1 kudos
- 1390 Views
- 5 replies
- 0 kudos
External Api not returning any response
import requestsurl = "https://example.com/api"headers = {"Authorization": "Bearer YOUR_TOKEN","Content-Type": "application/json"}Payload = json.dumps({json_data})response = requests.post(url, headers=headers, data=Payload)print(response.status_code)p...
- 1390 Views
- 5 replies
- 0 kudos
- 0 kudos
Can you DIM the two payload, it has to do with something on endpoint, since it is working on one payload fine, where in the other is not to the same endpoint.
- 0 kudos
- 1718 Views
- 1 replies
- 1 kudos
All my replies has been deleted and new one is being moderated (deleted) - why?
I noticed that when I Reply to post and trying to help solve community problem - my post are being either moderated (deleted) or just not being saved. Old post/replies has been deleted.Is there any reason for that?I kind of lost my will to participat...
- 1718 Views
- 1 replies
- 1 kudos
- 1 kudos
Currently, I'm also facing same issue, my comments are automatically deleted and also not receiving email for same, which is happening to specific post.
- 1 kudos
- 2617 Views
- 2 replies
- 2 kudos
Error accessing file from dbfs inside mlflow serve endpoint
Hi,I have mlflow model served using serverless GPU which takes audio file name as input and then file will be passed as parameter to huggiung face model inside predict method. But I am getting following errorHFValidationError(\nhuggingface_hub.utils....
- 2617 Views
- 2 replies
- 2 kudos
- 2 kudos
I have the same issue.I have a large file that I cannot access from an MLFlow service.Things I have tried (none of these work):Read-only from DBFS`dbfs:/myfolder/myfile.chroma` does not work`/dbfs/myfolder/myfile.chroma` does not workRead-only from U...
- 2 kudos
