cancel
Showing results for 
Search instead for 
Did you mean: 
Discussions
Engage in dynamic conversations covering diverse topics within the Databricks Community. Explore discussions on data engineering, machine learning, and more. Join the conversation and expand your knowledge base with insights from experts and peers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Browse the Community

Community Discussions

Engage in vibrant discussions covering diverse learning topics within the Databricks Community. Expl...

4350 Posts

Activity in Discussions

Mohanraj_Kandha
by > New Contributor
  • 7 Views
  • 0 replies
  • 0 kudos

unable to read file in workspace/user in free edition

I am unable to read the file in workspace/user in free edition, it was actually possible few weeks back but now it throws this error"[FAILED_READ_FILE.NO_HINT] Error while reading file dbfs:REDACTED_LOCAL_PART@outlook.com/BigMartSales.csv. SQLSTATE: ...

  • 7 Views
  • 0 replies
  • 0 kudos
abhijit007
by > New Contributor III
  • 53 Views
  • 2 replies
  • 1 kudos

Resolved! Databricks Asset Bundles capability for cross cloud migration

Hi everyone,We are planning a migration from Azure Databricks to GCP Databricks and would like to understand whether Databricks Asset Bundles (DAB) can be used to migrate workspace assets such as jobs, pipelines, notebooks, and custom serving endpoin...

  • 53 Views
  • 2 replies
  • 1 kudos
Latest Reply
abhijit007
New Contributor III
  • 1 kudos

@iyashk-DB Thanks for the details.. It helps.

  • 1 kudos
1 More Replies
greengil
by > New Contributor II
  • 356 Views
  • 10 replies
  • 2 kudos

Create function issue

Hello - I am following some online code to create a function as follows:-----------------------------------------CREATE OR REPLACE FUNCTION my_catalog.my_schema.insert_data_function(col1_value STRING,col2_value INT)RETURNS BOOLEANCOMMENT 'Inserts dat...

  • 356 Views
  • 10 replies
  • 2 kudos
Latest Reply
iyashk-DB
Databricks Employee
  • 2 kudos

In UC, the functions must be read-only; they cannot modify state (no INSERT, DELETE, MERGE, CREATE, VACUUM, etc). So I tried to create a PROCEDURE and call it; I was able to insert data into the table successfully. Unity Catalog tools are really jus...

  • 2 kudos
9 More Replies
nulltype
by > New Contributor II
  • 91 Views
  • 2 replies
  • 1 kudos

Managing Spark Declarative Pipelines Permissions

We are currently managing our permissions via Terraform (including cluster creation, UC governance, etc.). We have a specific `data_engineer` role, and we need everyone with this role to be able to view and manage all of our SDPs.The Issue: Currently...

  • 91 Views
  • 2 replies
  • 1 kudos
Latest Reply
nulltype
New Contributor II
  • 1 kudos

Our Solution: We moved job and pipeline permissions to DAB configuration files for streamlined enforcement. Terraform will remain the source of truth for workspace-level permissions only.

  • 1 kudos
1 More Replies
Suheb
by > Contributor
  • 40 Views
  • 1 replies
  • 0 kudos

What are the practical differences between bagging and boosting algorithms?

How are bagging and boosting different when you use them in real machine-learning projects?

  • 40 Views
  • 1 replies
  • 0 kudos
Latest Reply
iyashk-DB
Databricks Employee
  • 0 kudos

Bagging and boosting differ mainly in how they reduce error and when you’d choose them: Bagging (e.g., Random Forest) trains many models independently in parallel on different bootstrap samples to reduce variance, making it ideal for unstable, high-v...

  • 0 kudos
Hubert-Dudek
by Databricks MVP
  • 27 Views
  • 0 replies
  • 1 kudos

Databricks Advent Calendar 2025 #17

Replacing records for the entire date with newly arriving data for the given date is a typical design pattern. Now, thanks to simple REPLACE USING in Databricks, it is easier than ever! 

2025_17.png
  • 27 Views
  • 0 replies
  • 1 kudos
piotrsofts
by > New Contributor III
  • 36 Views
  • 1 replies
  • 2 kudos

Data segregated by Domain - how to do that

In the Databricks one demo, and even in official website I seehttps://www.databricks.com/blog/introducing-databricks-one"Domains", how to enable it ? Or add "Domain" to data?

piotrsofts_0-1765990740616.png
  • 36 Views
  • 1 replies
  • 2 kudos
Latest Reply
Hubert-Dudek
Databricks MVP
  • 2 kudos

I think domains are a new concept in Private Preview. If Unity Catalog has to become an enterprise catalog, then they will need to add that concept.

  • 2 kudos
piotrsofts
by > New Contributor III
  • 31 Views
  • 0 replies
  • 0 kudos

Accessing Knowledge Base from Databricks One

Is it possible to use Knowledge Assistant from Databricks one ? 

  • 31 Views
  • 0 replies
  • 0 kudos
RodrigoE
by > Visitor
  • 59 Views
  • 3 replies
  • 0 kudos

Vector search index initialization very slow

Hello,I am creating a vector search index and selected Compute embeddings for a delta table with 19M records.  Delta table has only two  columns: ID (selected as index) and Name (selected for embedding). Embedding model is databricks-gte-large-en.Ind...

Machine Learning
index
search
vector
vector index
Vector Search
  • 59 Views
  • 3 replies
  • 0 kudos
Latest Reply
iyashk-DB
Databricks Employee
  • 0 kudos

@RodrigoE please follow this document - https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/deploy-prov-throughput-foundation-model-apis#create-your-provisioned-throughput-endpoint-using-the-ui

  • 0 kudos
2 More Replies
seefoods
by > Valued Contributor
  • 33 Views
  • 0 replies
  • 0 kudos

spark conf for serveless jobs

Hello Guys, I use serveless on databricks Azure, so i have build a decorator which instanciate a SparkSession. My job use autolaoder / kafka using mode availableNow. Someone Knows which spark conf is required beacause i want to add it  ? Thanx import...

  • 33 Views
  • 0 replies
  • 0 kudos
kkirtigoel
by > Visitor
  • 26 Views
  • 0 replies
  • 0 kudos

Done the certification, but not received my badge

Hi Databricks Team,I have completed my AI Agentic Fundamental certificate course today and haven't received my badge for the same. Can you guys help me for this.

  • 26 Views
  • 0 replies
  • 0 kudos
Deelaka98
by > New Contributor
  • 63 Views
  • 1 replies
  • 1 kudos

My Databricks Fundamentals Accreditation badge didn't receive

I have completed Databricks Fundamentals Accreditation, but I haven't received the certification badge.Below attached here the certification, and my email address is: deelakawalagama@outlook.com.Also, I would like to add this badge to my LinkedIn pro...

2308_3_1356098_1765926433_Databricks - Generic.jpg
  • 63 Views
  • 1 replies
  • 1 kudos
Latest Reply
cert-ops
Databricks Employee
  • 1 kudos

Hello @Deelaka98 ,  Please check your credentials here. Please note that we cannot provide support via community (it is not advised that you post your email address here in community). Thanks & Regards,@cert-ops

  • 1 kudos
seefoods
by > Valued Contributor
  • 183 Views
  • 2 replies
  • 2 kudos

Resolved! setup databricks connect on VsCode and PyCharm

Hello Guyz,Someone Know what's is the best pratices to setup databricks connect for Pycharm and VsCode using Docker, Justfile and .env file Cordially, Seefoods

  • 183 Views
  • 2 replies
  • 2 kudos
Latest Reply
Gecofer
Contributor II
  • 2 kudos

Hi @seefoods!I’ve worked with Databricks Connect and VSCode in different projects, and although your question mentions Docker, Justfile and .env, the “best practices” really depend on what you’re trying to do. Here’s what has worked best for me:1.- D...

  • 2 kudos
1 More Replies
Joost1024
by > Visitor
  • 68 Views
  • 3 replies
  • 0 kudos

Read Array of Arrays of Objects JSON file using Spark

Hi Databricks Community! This is my first post in this forum, so I hope you can forgive me if it's not according to the forum best practices After lots of searching, I decided to share the peculiar issue I'm running into in this community.I try to lo...

  • 68 Views
  • 3 replies
  • 0 kudos
Latest Reply
Joost1024
Visitor
  • 0 kudos

I guess I was a bit over enthusiastic by accepting the answer.When I run the following on the single object array of arrays (as shown in the original post) I get a single row with column "value" and value null. from pyspark.sql import functions as F,...

  • 0 kudos
2 More Replies
rc10000
by > New Contributor
  • 65 Views
  • 2 replies
  • 3 kudos

Resolved! Data Bricks Engineer - DEA Exam vs Training

Hi, I love the Databricks resources but I'm a little confused on what training to take. My focus is studying and practicing for the Databricks Engineer Associate exam, but when I hear of the 'training', I'm not sure which training people are referrin...

  • 65 Views
  • 2 replies
  • 3 kudos
Latest Reply
Advika
Community Manager
  • 3 kudos

Hello @rc10000!+1 to what @Louis_Frolio  mentioned above.The Learning Plan is designed for users preparing for the Databricks Certified Data Engineer Associate and Professional exams. Also below are a few paths, depending on what you’re looking for: ...

  • 3 kudos
1 More Replies