cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Iguinrj11
by New Contributor II
  • 668 Views
  • 3 replies
  • 0 kudos

Resolved! DataBricks x Query Folding Power BI

I ran a native Power BI query in DataBricks in import mode and query folding was not enabled. No query folding? 

Iguinrj11_0-1737819675982.png
  • 668 Views
  • 3 replies
  • 0 kudos
Latest Reply
filipniziol
Esteemed Contributor
  • 0 kudos

Hi @Iguinrj11 ,The trick is to configure Databricks.Query instead of Databricks.Catalogs.Check this article and let us know if that helps:https://www.linkedin.com/pulse/query-folding-azure-databricks-tushar-desai/

  • 0 kudos
2 More Replies
tomvogel01
by New Contributor II
  • 383 Views
  • 2 replies
  • 0 kudos

Dynamic Bloom Filters for Inner Joins

I have a question regarding combining the use of Bloom filters with Liquid Clustering to further reduce the data read during a join/merge on top of dynamic file pruning. Testing both combined worked extremely well together for point queries. However ...

  • 383 Views
  • 2 replies
  • 0 kudos
Latest Reply
NandiniN
Databricks Employee
  • 0 kudos

We do not recommend Bloom filters Index on the Delta Tables as they have to be manually maintained.  If you prefer photon - please try predictive I/O with Liquid Clustering.

  • 0 kudos
1 More Replies
MadelynM
by Databricks Employee
  • 1490 Views
  • 9 replies
  • 0 kudos

Who's hiring? Latest Job Postings from the Databricks Community!

More than 10,000 organizations worldwide — including Block, Comcast, Condé Nast, Rivian, Shell and over 60% of the Fortune 500 — rely on the Databricks Data Intelligence Platform to take control of their data and put it to work with AI. Use this thre...

  • 1490 Views
  • 9 replies
  • 0 kudos
Latest Reply
MadelynM
Databricks Employee
  • 0 kudos

Cedar is seeking a seasoned Engineering Director to lead our Data Integration and Platforms team. This role will help us tackle challenges of scale, quality and insight as the company grows its market share and product scope.  Job Title:  Engineering...

  • 0 kudos
8 More Replies
ivvande
by New Contributor II
  • 399 Views
  • 4 replies
  • 0 kudos

Automate run as workflow parameter to default to current user

I am trying to run a workflow within Databricks. I have 2 workflows, workflow one which always runs as the service principal, as all data gets accessed and wrangled within this workflow, and workflow 2 which always defaults to the last run account. I...

ivvande_0-1737706760905.png
  • 399 Views
  • 4 replies
  • 0 kudos
Latest Reply
saurabh18cs
Valued Contributor III
  • 0 kudos

Hi, how are you expecting to achieve this? Do you want users who are manually triggering this workflow first update to their run_as? or you want to make this happen programatically?

  • 0 kudos
3 More Replies
subhadeep
by New Contributor II
  • 305 Views
  • 2 replies
  • 0 kudos

Create csv and upload on azure

Can some write a sql query , which queries a table like select * from stages.benefit , creates a csv and upload on azure 

  • 305 Views
  • 2 replies
  • 0 kudos
Latest Reply
hari-prasad
Valued Contributor II
  • 0 kudos

Hi @subhadeep ,You can achieve this in SQL similarly to how you write a dataframe into a table or blob path. We will create an external table pointing to the blob path or mounted blob path. Note that this table does not support ACID transactions and ...

  • 0 kudos
1 More Replies
idanyow
by New Contributor III
  • 793 Views
  • 10 replies
  • 2 kudos

01_demo_setup error

HelloI was following "Demo: Creating and Working with a Delta Table"while I have a community edition user.The first command in the Notebook is: %run ./setup/01_demo_setup But I got the following error:Notebook not found: Users/<my-email-was-here..>/s...

  • 793 Views
  • 10 replies
  • 2 kudos
Latest Reply
Isi
Contributor
  • 2 kudos

Hey!Sad news guys... if you go to Course Logistics Review you can read:"We are pleased to offer a version of this course that also contains hands-on practice via a Databricks Academy Labs subscription. With a Databricks Academy Labs subscription, you...

  • 2 kudos
9 More Replies
Rakeshch007
by New Contributor
  • 194 Views
  • 1 replies
  • 0 kudos

Databricks app giving 'upstream request timeout '

Hello all,We are developing an app which is based on flask, which is used to download logs from databricks dbfs location. For this useful case we are using databricks inbuilt App feature to deploy our app.While we pass a smaller file it is getting do...

  • 194 Views
  • 1 replies
  • 0 kudos
Latest Reply
Isi
Contributor
  • 0 kudos

Hey!It looks like the issue you’re facing might be related to the proxy timeout when downloading large files from DBFS. Since modifying the proxy settings might not be an option, there are a couple of alternative approaches you could consider to miti...

  • 0 kudos
haritashva31
by New Contributor
  • 3809 Views
  • 3 replies
  • 0 kudos

50%-off Databricks certification voucher

Hello Databricks Community Team, I am reaching out to inquire about the Databricks certification voucher promotion for completing the Databricks Learning Festival (Virtual) courses.I completed one of the Databricks Learning Festival courses July 2024...

  • 3809 Views
  • 3 replies
  • 0 kudos
Latest Reply
MARC9312
New Contributor II
  • 0 kudos

I have already finished the course, how do I get the discount?

  • 0 kudos
2 More Replies
Sudheer2
by New Contributor III
  • 261 Views
  • 1 replies
  • 0 kudos

User Unable to Access Key Vault Secrets Despite Role Assignment in Terraform

Hi All,I'm encountering an issue where a user is unable to access secrets in an Azure Key Vault, even though the user has been assigned the necessary roles using Terraform. Specifically, the user gets the following error when trying to access the sec...

  • 261 Views
  • 1 replies
  • 0 kudos
Latest Reply
mm41
New Contributor II
  • 0 kudos

Are they accessing the Key Vault directly and not through Databricks? If so, based on your Terraform code, they should be able to directly read Secrets in the Azure Key Vault.  You've configured the Key Vault with RBAC Authorization and assigned Key ...

  • 0 kudos
bvraravind
by New Contributor II
  • 130 Views
  • 1 replies
  • 0 kudos

Prevent users from running shell commands

Hi, is there any way to prevent users from running shell commands in Databricks notebooks? for example, "%%bash" I read that REVOKE EXECUTE ON SHELL command can be used. but i am unable to make it to work. Thanks in advance for any help.

  • 130 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @bvraravind, You can use this spark setting  "spark_conf.spark.databricks.repl.allowedLanguages": { "type": "fixed", "value": "python,sql" Over a cluster policy to prevent access to shell commands. https://docs.databricks.com/en/archive/compute/c...

  • 0 kudos
Sudheer2
by New Contributor III
  • 312 Views
  • 2 replies
  • 0 kudos

Terraform: Add Key Vault Administrator Role Assignment and Save Outputs to JSON Dynamically in Azure

Hi everyone,I am using Terraform to provision an OpenAI service and its modules along with a Key Vault in Azure. While the OpenAI service setup works as expected, I am facing two challenges:Role Assignment for Key VaultI need to assign the Key Vault ...

  • 312 Views
  • 2 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

For question two, you can use the local_file resource in Terraform: output "openai_api_type" {value = module.openai.api_type} output "openai_api_base" {value = module.openai.api_base} output "openai_api_version" {value = module.openai.api_version} ou...

  • 0 kudos
1 More Replies
thrinadhReddy
by New Contributor II
  • 2948 Views
  • 4 replies
  • 1 kudos

Cluster policy type

Hi guys,I am creating a cluster policy through json. "runtime_engine": {"type": "fixed""value": "PHOTON"}When I run the above code... PHOTON option is getting enabled but graying out... What would I specify in type field so that the photon option sho...

  • 2948 Views
  • 4 replies
  • 1 kudos
Latest Reply
Khalil_Robinson
New Contributor II
  • 1 kudos

Hey, If you change the type to "allowlist" and provide "PHOTON" and "STANDARD" as options that should fix your issue.here is an example: "runtime_engine": {"type": "allowlist""value": ["PHOTON", "STANDARD"]"defaultValue": "PHOTON"}

  • 1 kudos
3 More Replies
sshukla
by New Contributor III
  • 385 Views
  • 5 replies
  • 0 kudos

External Api not returning any response

import requestsurl = "https://example.com/api"headers = {"Authorization": "Bearer YOUR_TOKEN","Content-Type": "application/json"}Payload = json.dumps({json_data})response = requests.post(url, headers=headers, data=Payload)print(response.status_code)p...

  • 385 Views
  • 5 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Can you DIM the two payload, it has to do with something on endpoint, since it is working on one payload fine, where in the other is not to the same endpoint.

  • 0 kudos
4 More Replies
Wojciech_BUK
by Valued Contributor III
  • 1557 Views
  • 1 replies
  • 1 kudos

All my replies has been deleted and new one is being moderated (deleted) - why?

I noticed that when I Reply to post and trying to help solve community problem - my post are being either moderated (deleted) or just not being saved. Old post/replies has been deleted.Is there any reason for that?I kind of lost my will to participat...

  • 1557 Views
  • 1 replies
  • 1 kudos
Latest Reply
hari-prasad
Valued Contributor II
  • 1 kudos

Currently, I'm also facing same issue, my comments are automatically deleted and also not receiving email for same, which is happening to specific post.

  • 1 kudos
sanjay
by Valued Contributor II
  • 2362 Views
  • 2 replies
  • 1 kudos

Error accessing file from dbfs inside mlflow serve endpoint

Hi,I have mlflow model served using serverless GPU which takes audio file name as input and then file will be passed as parameter to huggiung face model inside predict method. But I am getting following errorHFValidationError(\nhuggingface_hub.utils....

  • 2362 Views
  • 2 replies
  • 1 kudos
Latest Reply
txti
New Contributor III
  • 1 kudos

I have the same issue.I have a large file that I cannot access from an MLFlow service.Things I have tried (none of these work):Read-only from DBFS`dbfs:/myfolder/myfile.chroma` does not work`/dbfs/myfolder/myfile.chroma` does not workRead-only from U...

  • 1 kudos
1 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Top Kudoed Authors