Announcing: Workflows!

User16835756816 — Tue, 10 May 2022 19:19:19 GMT

🎉 Databricks is excited to announce the general availability of Databricks Workflows to you, our community. 🎉

Databricks Workflows is the fully managed lakehouse orchestration service for all your teams to build reliable data, analytics, and AI workflows on any cloud.

Orchestrate any combination of notebooks, SQL, Spark, ML models and dbt as a Jobs workflow, including calls to other systems. Build ETL pipelines that are automatically managed, including ingestion and lineage, using Delta Live Tables. Databricks Workflows is available on GCP, AWS and Azure, giving you full flexibility and cloud independence.

All of you, our community members, can help power your business with fresh, reliable data and insights.

Getting started is easy

1.) Simply log in to your workspace

2.) or sign-up for a trial!

Learn it

To learn more, check out these resources created for Workflows

watch this Workflows demo
read the blog for details on a 7-task workflow
check out our Workflows documentation
feel free to contact us.

Join us live!

If you’re interested in learning about Workflows live from Databricks experts, watch our session at the Data & AI Summit this year.

Start the conversation! Drop your questions, feedback and tips below once you get going with Workflows! 👇

Re: Announcing: Workflows!

PawanShukla — Thu, 26 May 2022 09:14:28 GMT

I am trying to run the Workflow Pipeline with smaple code shared in getting start.. and getting the below error :

DataPlaneException: Failed to start the DLT service on cluster 0526-084319-7hucy1np. Please check the stack trace below or driver logs for more details.

java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider org.apache.spark.sql.eventhubs.EventHubsSourceProvider could not be instantiated

at java.util.ServiceLoader.fail(ServiceLoader.java:232)

topic Announcing: Workflows! in Data Engineering

Announcing: Workflows!

Re: Announcing: Workflows!