cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Next-Level Interactivity in AI/BI Dashboards

As recently announced at this year’s Data and AI Summit, Databricks AI/BI democratizes business intelligence and analytics across your organization with highly visual and interactive low-code AI/BI Dashboards and no-code conversational analytics powe...

  • 200 Views
  • 0 replies
  • 2 kudos
Tuesday
Announcing advanced security and governance in Mosaic AI Gateway

We are excited to introduce several powerful new capabilities to Mosaic AI Gateway, designed to help our customers accelerate their AI initiatives with even greater simplicity, security, and governance. Read more here

  • 116 Views
  • 1 replies
  • 2 kudos
Tuesday
Submit your feedback and win a $25 gift card!

Your feedback is crucial to us and directly influences how we innovate and improve our customer experience.  To share your success and feedback with Databricks, please  visit this link.  The first 20 people to submit a completed survey response will ...

  • 516 Views
  • 0 replies
  • 1 kudos
a week ago
Databricks Unity Catalog Workshop

Thursday, September 12, 2024 | 9:00 AM-10:35 AM PT Effective governance is essential to democratizing data and AI in the age of LLMs and generative AI. Enterprises face challenges with fragmented governance across databases, warehouses, and lakes, c...

  • 1658 Views
  • 1 replies
  • 2 kudos
3 weeks ago
Data + AI World Tour 2024

Over 12,000 companies have adopted data intelligence. Join a tour stop near you to discover how Databricks can unlock your data and AI too. Find your city now!  Explore technologies Explore the latest advancements in the Databricks Data Intellige...

  • 2652 Views
  • 0 replies
  • 5 kudos
07-23-2024

Community Activity

Nishat
by > • New Contributor
  • 57 Views
  • 1 replies
  • 0 kudos

Unity Catalog

Can MLflow Experiments be incorporated into Unity Catalog (may be in volume or somewhere in unity catalog) similar to models and feature tables?

Community Platform Discussions
Reply
  • 57 Views
  • 1 replies
  • 0 kudos
Reply
Latest Reply
Walter_C
Honored Contributor
  • 0 kudos

As of now Databricks supports DBFS, S3, and Azure Blob storage artifact locations. Our teams are working on intruducing Volumes as repository which might be coming soon 

  • 0 kudos
Jaku6
by > • New Contributor
  • 85 Views
  • 1 replies
  • 1 kudos

Run now with different parameters doesn't pass parameter to pipeline tasks

I have a job with some tasks. Some of the tasks are pipeline_task's some are notebook_task's.When I run the job with "Run now with different parameters" and enter a new key-value, I see that the key-value is available in the notebook_task's with dbut...

Data Engineering
Reply
  • 85 Views
  • 1 replies
  • 1 kudos
Reply
Latest Reply
Walter_C
Honored Contributor
  • 1 kudos

As per docs it seems that pipeline task type is currently not supported to pass parameters: https://docs.databricks.com/en/jobs/create-run-jobs.html#pass-parameters-to-a-databricks-job-taskYou could create a notebook task that runs before your pipeli...

  • 1 kudos
data-digger
by > • Visitor
  • 29 Views
  • 0 replies
  • 0 kudos

Hide function defination

Hi - I would like to hide function definition from user, what is the best way to do it in unity catalog

Data Governance
Reply
  • 29 Views
  • 0 replies
  • 0 kudos
Reply
eric-c
by > • New Contributor
  • 44 Views
  • 1 replies
  • 0 kudos

Alert ACL's not available

The list of available ACL's for alerts suggests there's a way to set "can run" and "can manage" permissions but neither the REST API nor the Databricks CLI show options for setting permissions. Is there a way to set acl's on alerts? If so, how?Alert ...

Administration & Architecture
Reply
  • 44 Views
  • 1 replies
  • 0 kudos
Reply
Latest Reply
Walter_C
Honored Contributor
  • 0 kudos

The API call needed to set the ACLs for an alert and other object types as (alerts | dashboards | data_sources | queries) is:https://docs.databricks.com/api/workspace/dbsqlpermissions/set

  • 0 kudos
cadull
by > • New Contributor II
  • 52 Views
  • 1 replies
  • 1 kudos

Permission Issue with IDENTIFIER clause

Hi all,we are parameterizing environment specific catalog names (like `mycatalog_dev` vs. `mycatalog_prd`) in Lakeview dashboard queries like this:SELECT * FROM IDENTIFIER(:catalog_name || '.myschema.mytable')Which works fine in most cases. We have o...

Data Engineering
Reply
  • 52 Views
  • 1 replies
  • 1 kudos
Reply
Latest Reply
madams
Visitor
  • 1 kudos

I've had quite a bit of fun with UC and view permissions.  I don't think this is specific to using the IDENTIFIER() function, but I suspect it's related to UC permissions.  What you'll need to ensure:The user or group who owns the view on catalog_b h...

  • 1 kudos
tdk
by > • New Contributor
  • 18 Views
  • 1 replies
  • 0 kudos

Cannot install jar to cluster: invalid authority.

Hi allI want to access on-prem Oracle Database data from the python notebooks. However, the install of the jar (ojdbc8.jar) results in an error, which occurs while the cluster is starting up.The error message:"Library installation attempted on the dr...

Data Engineering
Reply
  • 18 Views
  • 1 replies
  • 0 kudos
Reply
Latest Reply
Walter_C
Honored Contributor
  • 0 kudos

The error message suggests that the jar file located at abfss:/jars/ojdbc8.jar has an invalid authority. This could be due to a number of reasons such as incorrect file path, insufficient permissions, or network restrictions. Here are a few steps you...

  • 0 kudos
semsim
by > • Contributor
  • 15 Views
  • 0 replies
  • 0 kudos

Can't query Legacy Serving Endpoint

Hi,I was able to deploy an endpoint using legacy serving (It's the only option we have to deploy endpoints in DB). Now I am having trouble querying the endpoint itself. When I try to query it I get the following error:    Here is the code I am using ...

semsim_0-1726245119742.png
Machine Learning
Reply
  • 15 Views
  • 0 replies
  • 0 kudos
Reply
FedericoRaimond
by > • New Contributor
  • 167 Views
  • 7 replies
  • 1 kudos

Azure Databricks Workflows with Git Integration

Hello,I receive a very weird error when attempting to connect my workflows tasks to a remote git on azure devops.As per documentation: "For a Git repository, the path relative to the repository root."Then, I use directly the name of the notebook file...

Data Engineering
Reply
  • 167 Views
  • 7 replies
  • 1 kudos
Reply
Latest Reply
madams
Visitor
  • 1 kudos

Ah, I have had that same error before when cloning from git.  I'm guessing you got the Repo URL by hitting the "clone" button in ADO and copy/pasting it in to Databricks.  One thing I've done in the past is in the Repository URL, remove the "org@" pa...

  • 1 kudos
6 More Replies
DatabricksGuide
by • Community Manager
  • 3553 Views
  • 1 replies
  • 2 kudos

Getting Started with Databricks - Build a simple Lakehouse analytics pipeline

Getting started with Databricks - Build a simple Lakehouse analytics pipeline The demo example in this guide illustrates a lakehouse analytics pipeline using the well-known NYC taxi trip dataset. This public dataset is also available at Kaggle.  This...

DatabricksGuide_0-1723742834161.png DatabricksGuide_1-1723742833943.png DatabricksGuide_2-1723742833913.png
Get Started Guides
Reply
  • 3553 Views
  • 1 replies
  • 2 kudos
Reply
Latest Reply
DBQUEST
New Contributor
  • 2 kudos

"Excellent exercise! The most challenging part was setting up the environment using the Azure free account. However, now that I have access to all the functionalities, I am well-equipped to continue training and building my skills with Databricks. Lo...

  • 2 kudos
Visakh_Vijayan
by > • New Contributor
  • 59 Views
  • 3 replies
  • 0 kudos

Uninstall whl file from databricks cluster via CLI

Hello, we have a need to uninstall older versions of whl files from personal cluster via databricks CLI - could you please provide the exact command to be used here. We tried with many found on the documentations but none of them worked to do the act...

Machine Learning
Reply
  • 59 Views
  • 3 replies
  • 0 kudos
Reply
Latest Reply
szymon_dybczak
Contributor
  • 0 kudos

Hi @Visakh_Vijayan ,Did you try to use databricks libraries uninstall? It's exactly crafted for this purposedatabricks libraries uninstall --json YOUR_JSON_WITH_REQUEST_BODYAlso, when you uninstall a library from a cluster, the library is removed onl...

  • 0 kudos
2 More Replies
ishmaelamin
by > • New Contributor
  • 46 Views
  • 1 replies
  • 0 kudos

Databricks Academy Labs Subscription (Partners)

Is the subscription to the "Databricks Academy Labs Subscription (Partners)" useful, given that I have access to the Databricks platform through Visual Studio? Does the ecosystem perform better for the lab samples.  I ask this question because when I...

Training offerings
Reply
  • 46 Views
  • 1 replies
  • 0 kudos
Reply
Latest Reply
ishmaelamin
New Contributor
  • 0 kudos

...

  • 0 kudos
hyedesign
by > • New Contributor II
  • 1571 Views
  • 6 replies
  • 0 kudos

Getting SparkConnectGrpcException: (java.io.EOFException) error when using foreachBatch

Hello, I am trying to write a simple upsert statement following the steps in the tutorials. here is what my code looks like:from pyspark.sql import functions as Fdef upsert_source_one(self df_source = spark.readStream.format("delta").table(self.so...

Data Engineering
pyspark
Reply
  • 1571 Views
  • 6 replies
  • 0 kudos
Reply
Latest Reply
seans
New Contributor III
  • 0 kudos

I figured out the error is hiding an underlying issue with the code, which you can get to if you deploy the bundle (if you are using asset bundles) and run from a notebook in a browser.So the issue is more about the debugger not being able to stop on...

  • 0 kudos
5 More Replies
Umair_Hussain
by > • Visitor
  • 28 Views
  • 0 replies
  • 0 kudos

Databricks exam got suspended

My Databricks Certified Data Engineer Associate exam got suspended on 13th Sep, today and it is in progress state.Support person asked me to show the full table and I have shown the entire room and table. And I am not sure why the support person stil...

Certifications
Reply
  • 28 Views
  • 0 replies
  • 0 kudos
Reply
juan_barreto
by > • New Contributor
  • 133 Views
  • 1 replies
  • 1 kudos

Problem with dropDuplicates in Databricks runtime 15.4LTS

Hi,I'm testing the latest version of the databricks runtime but I'm getting errors doing a simple dropDuplicates.Using the following codedata = spark.read.table("some_table") data.dropDuplicates(subset=['SOME_COLUMN']).count() I'm getting this error....

juan_barreto_0-1726153266526.png
Data Engineering
Reply
  • 133 Views
  • 1 replies
  • 1 kudos
Reply
Latest Reply
kellys
Visitor
  • 1 kudos

Wanted to add to this thread. Seeing the same issue. This appears to be recent problem.

  • 1 kudos
Kaviprakash_S
by > • New Contributor III
  • 10487 Views
  • 7 replies
  • 1 kudos

Resolved! Free voucher for data engineer associate certification

Hi,I'm looking for a free voucher to take up the Data Engineer Associate certification. Could anyone guide please ? 

Certifications
Reply
  • 10487 Views
  • 7 replies
  • 1 kudos
Reply
Latest Reply
Cert-Team
Esteemed Contributor
  • 1 kudos

Doesn't look like it, sorry! https://community.databricks.com/t5/events/databricks-learning-festival-virtual-10-october-31-october/ec-p/87306#M502

  • 1 kudos
6 More Replies
Load more

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Announcements
Top Kudoed Authors
View all

Latest from our Blog

GenAI is for Data Engineers Too

Large Language Models (LLMs) or GenAI have taken center stage in their ability to summarize large text, write code, articles, poems, explain complex topics, and power now ubiquitous chatbots. Data Eng...

350Views 1kudos

6 Migration Mistakes You Don’t Want To Make: Part 1

Congratulations! Your team has just signed a contract with Databricks and you’re ready to start your first project.  Maybe.  Statistically, most of you will need to migrate off your previous platform ...

1727Views 4kudos

Data Validation: The Last Mile of Migration

Introduction Enterprise Data Warehouse (EDW) Migration use cases are always complex in nature. Data architects and Data engineers spend the majority of time in analyzing the source data, identifying t...

456Views 4kudos
View all