cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Announcing: Workflows!

User16835756816
Valued Contributor

🎉 Databricks is excited to announce the general availability of Databricks Workflows to you, our community. 🎉

Databricks Workflows is the fully managed lakehouse orchestration service for all your teams to build reliable data, analytics, and AI workflows on any cloud.

Orchestrate any combination of notebooks, SQL, Spark, ML models and dbt as a Jobs workflow, including calls to other systems. Build ETL pipelines that are automatically managed, including ingestion and lineage, using Delta Live Tables. Databricks Workflows is available on GCP, AWS and Azure, giving you full flexibility and cloud independence. 

All of you, our community members, can help power your business with fresh, reliable data and insights. 

Getting started is easy

1.) Simply log in to your workspace

2.) or sign-up for a trial!

Learn it

To learn more, check out these resources created for Workflows

Join us live!

If you’re interested in learning about Workflows live from Databricks experts, watch our session at the Data & AI Summit this year.

Start the conversation! Drop your questions, feedback and tips below once you get going with Workflows! 👇

4 REPLIES 4

PawanShukla
New Contributor III

I am trying to run the Workflow Pipeline with smaple code shared in getting start.. and getting the below error :

DataPlaneException: Failed to start the DLT service on cluster 0526-084319-7hucy1np. Please check the stack trace below or driver logs for more details.

java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider org.apache.spark.sql.eventhubs.EventHubsSourceProvider could not be instantiated

at java.util.ServiceLoader.fail(ServiceLoader.java:232)

Kaniz
Community Manager
Community Manager

Hi @Pawan Shukla​, Can you share the sample code where you've encountered the error?

Kaniz
Community Manager
Community Manager

Hi @Pawan Shukla​, , This is an issue with how your Databricks cluster is configured. Please detach your cluster and restart your cluster. Thanks.

Kaniz
Community Manager
Community Manager

Hi @Pawan Shukla​, We haven't heard from you since my last response, and I was checking back to see if my suggestions helped you. Or else, If you have any solution, please share it with the community as it can be helpful to others.

Also, please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.