I have a multi-part question around Databricks integration with Splunk?

[r_van_niekerk]

Use Case Background

We have an ongoing SecOps project going live here in 4 weeks. We have set up a Splunk to monitor syslogs logs and want to integrate this with Delta. Our forwarder collect the data from remote machines then forwards data to the index in real-time; our indexer processes the incoming stream in real-time and we typically query that data directly in vai the Splunk UI/Search Head.

We would like to provide our end users the ability to store historical logs in Delta; then query those directly logs via the Databricks UI/Notebooks/Databricks SQL.

Question

1. Whether there are any example notebooks or documentation/tips on Splunk integration with Databricks?
2. Whether you can query our logs directly via Databricks?

Thank you!

[aladda]

Yes. Please see the following post for details - https://uat-databrickspartner.cs165.force.com/forums/s/question/0D56s00000CxDvqCAF/does-databricks-i...

[aladda]

The Databricks Add-on for Splunk built as part of Databricks Labs can be leveraged for Splunk integration

• It’s a bi-directional framework that allows for in-place querying of data in databricks from within Splunk by running queries, notebooks or jobs so you don’t have to move the data and still have access to it from within. Docs are here - https://github.com/databrickslabs/splunk-integration#Documentation