Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
Showing results for 
Search instead for 
Did you mean: 

Announcing: Workflows!

Valued Contributor

🎉 Databricks is excited to announce the general availability of Databricks Workflows to you, our community. 🎉

Databricks Workflows is the fully managed lakehouse orchestration service for all your teams to build reliable data, analytics, and AI workflows on any cloud.

Orchestrate any combination of notebooks, SQL, Spark, ML models and dbt as a Jobs workflow, including calls to other systems. Build ETL pipelines that are automatically managed, including ingestion and lineage, using Delta Live Tables. Databricks Workflows is available on GCP, AWS and Azure, giving you full flexibility and cloud independence. 

All of you, our community members, can help power your business with fresh, reliable data and insights. 

Getting started is easy

1.) Simply log in to your workspace

2.) or sign-up for a trial!

Learn it

To learn more, check out these resources created for Workflows

Join us live!

If you’re interested in learning about Workflows live from Databricks experts, watch our session at the Data & AI Summit this year.

Start the conversation! Drop your questions, feedback and tips below once you get going with Workflows! 👇


New Contributor III

I am trying to run the Workflow Pipeline with smaple code shared in getting start.. and getting the below error :

DataPlaneException: Failed to start the DLT service on cluster 0526-084319-7hucy1np. Please check the stack trace below or driver logs for more details.

java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider org.apache.spark.sql.eventhubs.EventHubsSourceProvider could not be instantiated


Hi @Pawan Shukla​, Can you share the sample code where you've encountered the error?

Hi @Pawan Shukla​, , This is an issue with how your Databricks cluster is configured. Please detach your cluster and restart your cluster. Thanks.

Hi @Pawan Shukla​, We haven't heard from you since my last response, and I was checking back to see if my suggestions helped you. Or else, If you have any solution, please share it with the community as it can be helpful to others.

Also, please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!