cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 
What’s New With Databricks Assistant?

Over the past few months, we’ve been gathering your feedback and focusing on both the quality of Databricks Assistant’s responses and the overall user experience. Today, we're excited to showcase a more advanced Databricks Assistant, packed with powe...

  • 473 Views
  • 0 replies
  • 2 kudos
2 weeks ago
Databricks Community Champion - October 2024 - Filip Niziol

Meet Filip Niziol, a Senior Data Engineer at Digiterre and an active contributor to the Databricks Community. With a rich background in data architecture and a passion for building scalable data solutions, Filip shares insights on his career, Databri...

  • 324 Views
  • 3 replies
  • 5 kudos
Monday
Become Our Next Monthly Community Champion!

Each month, we recognize one outstanding member of our Databricks Community who has made a real impact. Whether you share solutions, spark discussions, or help others, this is your chance to get featured! What’s a Monthly Community Champion? Our ...

  • 202 Views
  • 0 replies
  • 4 kudos
Monday
Introducing Simple, Fast, and Scalable Batch LLM Inference on Mosaic AI Model Serving

Over the years, organizations have amassed a vast amount of unstructured text data—documents, reports, and emails—but extracting meaningful insights has remained a challenge. Large Language Models (LLMs) now offer a scalable way to analyze this data,...

  • 490 Views
  • 0 replies
  • 1 kudos
2 weeks ago
Databricks Migration Strategy: Lessons Learned

Migrating your data warehouse workloads is one of the most challenging yet essential tasks for any organization. Whether the motivation is the growth of your business and scalability requirements or reducing the high license and hardware cost of your...

  • 1218 Views
  • 0 replies
  • 2 kudos
2 weeks ago

Community Activity

Brad
by > Contributor
  • 107 Views
  • 3 replies
  • 0 kudos

why latestOffset and getBatch takes so long time

Hi team,Kinesis -> delta table raw -> job with trigger=availableNow -> delta table target. The Kinesis->delta table raw is running continuously. The job is daily with trigger=availableNow. The job reads from raw, do some transformation, and run a MER...

Brad_0-1729034240965.png
  • 107 Views
  • 3 replies
  • 0 kudos
Latest Reply
Brad
Contributor
  • 0 kudos

When delta table tries to restore a version, it can find the closest checkpoint and apply versions up to the target. However, delta table needs to read and apply many versions to get changes between 2 versions. There are no compact file can be used. ...

  • 0 kudos
2 More Replies
sathya08
by > New Contributor
  • 9 Views
  • 0 replies
  • 0 kudos

Databricks Python function achieving Parallelism

Hello everyone,I have a very basic question wrt Databricks spark parallelism.I have a python function within a for loop, so I believe this is running sequentially.Databricks cluster is enabled with Photon and with Spark 15x, does that mean the driver...

  • 9 Views
  • 0 replies
  • 0 kudos
Steven_Roy
by > New Contributor II
  • 10 Views
  • 0 replies
  • 0 kudos

Just Passed Databricks-Machine-Learning-Professional exam

Hi guys,I have the ML Professional exam scehduled for later this month and while I can find many resources, practice exams, and posts related to the ML Associate exam, I'm having trouble finding the same for the Professional exam.Anyone happen to hav...

  • 10 Views
  • 0 replies
  • 0 kudos
Sanjeev
by > New Contributor II
  • 41 Views
  • 2 replies
  • 1 kudos

Unverified Commits via Databricks Repos: Seeking Solution for GitHub Verification

The team is integrating Databricks Repos with Personal Access Tokens (PAT) to commit code directly to GitHub. Our organization requires signed commits for verification purposes.Issue: When committing via Databricks Repos, the commits appear as unveri...

Data Engineering
data engineering
  • 41 Views
  • 2 replies
  • 1 kudos
Latest Reply
Sanjeev
New Contributor II
  • 1 kudos

Can you please share the link to this doc DB-I-3082. I couldn't find it.

  • 1 kudos
1 More Replies
edisionthomas
by > New Contributor
  • 39 Views
  • 1 replies
  • 0 kudos

Is it possible to configure Databricks to leverage an AI PC’s NVMe storage for faster I/O?

Hello Databricks Community,I’m working on improving the performance of my Databricks workflows and exploring different storage options. I have an AI PC with NVMe storage, and I want to know if it’s possible to set up Databricks to use this storage fo...

  • 39 Views
  • 1 replies
  • 0 kudos
Latest Reply
BigRoux
Databricks Employee
  • 0 kudos

This is not how you should be using Databricks.  Databricks is SaaS/Paas and it's main operating model is operate on data stored in Cloud Object Storage (GCS, S3, ADLS Gen 2).  It is optimized for cloud storage, morever it is designed to be very fast...

  • 0 kudos
DB_Learner_3
by > New Contributor
  • 44 Views
  • 1 replies
  • 0 kudos

Help needed with automated file ingestion process for full refresh mode using autoloader preferable

Description:As a data engineer, I need to implement an automated process to ingest data from multiple files in a subdirectory and create corresponding bronze tables. This process should handle full file refreshes and consider strategies to limit the ...

  • 44 Views
  • 1 replies
  • 0 kudos
Latest Reply
BigRoux
Databricks Employee
  • 0 kudos

What do you mean by "full file refreshes"?  Does this refer to the fact that file names will be reused?

  • 0 kudos
ashish_modi
by > New Contributor II
  • 104 Views
  • 3 replies
  • 1 kudos

cloud_infra_costs

I was looking at the system catalog and realized that there is an empty table called cloud_infra_costs. Could you tell me what is this for and why it is empty?  

ashish_modi_0-1730131553537.png
  • 104 Views
  • 3 replies
  • 1 kudos
Latest Reply
ashish_modi
New Contributor II
  • 1 kudos

Thanks for replying. This makes sense. Any idea why it is empty and what to do to populate this?

  • 1 kudos
2 More Replies
josh_melton
by Databricks Employee
  • 1973 Views
  • 2 replies
  • 2 kudos

The New Data Analyst: Transforming BI in the Age of AI with Databricks Genie

Introduction Data and AI media will try to convince you of two terribly misguided ideas about AI’s impact on analytics. First of all, there’s the idea that prompt engineers who know just the right phrase to unlock the secret powers of generative AI w...

josh_melton_0-1724170554601.png josh_melton_1-1724170554693.png josh_melton_2-1724170554643.png josh_melton_3-1724170554662.png
  • 1973 Views
  • 2 replies
  • 2 kudos
Latest Reply
josh_melton
Databricks Employee
  • 2 kudos

I missed this question, sorry - Genie API is in private preview, reach out to your account team for access

  • 2 kudos
1 More Replies
Benni
by > New Contributor
  • 70 Views
  • 2 replies
  • 0 kudos

UC, pyodbc, Shared Cluster, and :Can't open lib 'ODBC Driver 17 for SQL Server' : file not found

Hey Databricks!    Trying to use the pyodbc init script in a Volume in UC on a shared compute cluster but receive error: "[01000] [unixODBC][Driver Manager]Can't open lib 'ODBC Driver 17 for SQL Server' : file not found (0) (SQLDriverConnect)"). I fo...

Benni_0-1730842808643.png Benni_1-1730842895078.png
  • 70 Views
  • 2 replies
  • 0 kudos
Latest Reply
agallard
New Contributor III
  • 0 kudos

Hi @Benni ,Ensure that the ODBC Driver 17 for SQL Server is installed on the Databricks cluster, you can verify this by running the following command in a notebook cell on the cluster to check if the driver is available in the system:!ls /opt/microso...

  • 0 kudos
1 More Replies
takak
by > Visitor
  • 32 Views
  • 1 replies
  • 0 kudos

Cannot downgrade workspace object permissions using API

Hi!I'd like to restrict some users' permissions using REST API and got an issue while trying to update a permission on 'directories'.I'm trying to set a user's permission on their default username folder in the workspace to 'can edit' so that they ca...

  • 32 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @takak, Greetings from Databricks! What is the REST API you are making the call to? Looks like this might not be supported programmatically, but will try to test it internally. it appears that the CAN_MANAGE permission is a higher-level permission...

  • 0 kudos
Frank_Kennedy
by > New Contributor II
  • 82 Views
  • 2 replies
  • 0 kudos

I want to play from my phone

Maybe someone can advise something so that you can play on the road or on vacation

  • 82 Views
  • 2 replies
  • 0 kudos
Latest Reply
Richardooper
  • 0 kudos

Hey there! If you're looking to use your phone for some fun, it's totally possible and actually pretty convenient! For ios users, there are plenty of options to enjoy a variety of activities with ease. One platform you might want to check out is Jeet...

  • 0 kudos
1 More Replies
vyasakhilesh
by > Visitor
  • 30 Views
  • 1 replies
  • 0 kudos

Error Creating Table from delta location dbfs

[UC_FILE_SCHEME_FOR_TABLE_CREATION_NOT_SUPPORTED] Creating table in Unity Catalog with file scheme dbfs is not supported. Instead, please create a federated data source connection using the CREATE CONNECTION command for the same table provider, then ...

  • 30 Views
  • 1 replies
  • 0 kudos
Latest Reply
agallard
New Contributor III
  • 0 kudos

Hi @vyasakhilesh,The error you're seeing, [UC_FILE_SCHEME_FOR_TABLE_CREATION_NOT_SUPPORTED], occurs because Unity Catalog in Databricks does not support creating tables directly from DBFS (Databricks File System) locations.In this case, you're trying...

  • 0 kudos
SethParker
by > New Contributor III
  • 101 Views
  • 6 replies
  • 0 kudos

Resolved! SQL View Formatting in Catalog - Can you turn it off?

It appears as though Databricks now formats SQL View definitions when showing them in the Catalog.  Our solution is based on views, and we have comment tags in those views.  We format these views so that it is easy for us to find and update parts of ...

SethParker_0-1730757882698.png SethParker_1-1730757945105.png
  • 101 Views
  • 6 replies
  • 0 kudos
Latest Reply
SethParker
New Contributor III
  • 0 kudos

Thank you!  I will submit that request.In case anyone else stumbles upon this post, here is a function you can add that will return the view definition from information_schema, unformatted with an ALTER statement at the top:DROP FUNCTION IF EXISTS <c...

  • 0 kudos
5 More Replies
Kaushi
by > New Contributor
  • 34 Views
  • 1 replies
  • 0 kudos

Cannot able to attend the quiz after databricks lakehouse fundamentals

Hi, I have already finished my databricks fundamentals quiz in my partner academy account. I got the certificate (There was no badge logo in the certificate), and also not yet received my badge in my mail..its been 2 days since i completed my exam.Ki...

ss-1.png ss-2.png
  • 34 Views
  • 1 replies
  • 0 kudos
Latest Reply
Advika
Databricks Employee
  • 0 kudos

Hello @Kaushi, Please file a ticket with our support team and allow the support team 24-48 hours for a resolution.  

  • 0 kudos
priti18
by > New Contributor
  • 54 Views
  • 1 replies
  • 0 kudos

Cleared the exam , however didnt receive the certificate yet after 48 hours

Cleared the exam for databricks data engg associate   , however I didnt receive the certificate yet even now 2 days are already passed 

  • 54 Views
  • 1 replies
  • 0 kudos
Latest Reply
Advika
Databricks Employee
  • 0 kudos

@priti18 Please file a ticket with our support team and allow the support team 24-48 hours for a resolution.

  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Featured Events
Featured Event - Chennai Data Day

Chennai Data Day

Join the Chennai User Group Meetup to connect with fellow data enthusiasts and learn about the latest in data analytics.

View Event Details
Top Kudoed Authors
Read Databricks Data Intelligence Platform reviews on G2

Latest from our Blog

Getting started with fine-tuning on Databricks

Authors: Ellen Hirt, Narjes Majdoub, Giran Moodley Contents ContentsIntroductionSection 1: Preparing the EnvironmentSection 2: Step-by-Step Instructions1. Data preparation2. Model initialisation and t...

164Views 1kudos

Databricks observability using Grafana and Prometheus

Databricks Observability using Grafana and Prometheus As software systems scale, the amount of data they process and the work they do grows with them. This is not limited only to cloud-native services...

1553Views 3kudos