Available Connectors for ServiceNow to Databricks

[Phani1]

Hi Team,

What are the available connectors to bring data and metadata from ServiceNow to Databricks, and what are the best options/best practices for integrating ServiceNow with Databricks ?

Regards,

Phani

[szymon_dybczak]

Hi @Phani1 ,

There's official databricks service now connection. You can read about it at below link:

Configure ServiceNow for Databricks ingestion - Azure Databricks | Microsoft Learn

And here you have example of how to create ingestion pipeline using that connector:

Create a ServiceNow ingestion pipeline - Azure Databricks | Microsoft Learn

[Phani1]

Thanks for your response and providing the various options.

Databricks offering, Lake Flow feature, connects to Service Now directly and allows for pipeline creation using low-code and no-code options. However, I need suggestions on creating a common pipeline for 200+ tables in Lake Flow Connect. Additionally, can Lake Flow Connect handle and process huge amounts of historical data?

Regards,

Phani

[szymon_dybczak]

Hi @Phani1 ,

There is a limitation of this connector when it comes to number of tables per pipeline. So, there is a limit of 250 tables per pipeline. 
But that's easy to overcome - to ingest more than 250 tables, create multiple pipelines.

There' so limitation regarding loading historical data, In fact, this connector supports incremental loading pattern, so in theory it looks like perfect candidate to use as a servicenow integration tool.

But, check following list of limitations for this lakeflow connect connector to be sure that it's suitable for your use case:

ServiceNow connector limitations - Azure Databricks | Microsoft Learn