topic Real time data quality validation (Streaming data ingestion) in Data Governance

Real time data quality validation (Streaming data ingestion)

laksh — Thu, 02 Mar 2023 01:28:44 GMT

I was wondering how the Unity Catalog would help in data quality validations for real time (streaming data) data ingestion.

Re: Real time data quality validation (Streaming data ingestion)

laksh — Thu, 02 Mar 2023 17:30:46 GMT

Hi Debayan,

I reviewed your link, but it does not speak about any Data quality checks for Realtime/stream data sets. can you share information on that. We are currently reviewing what we can accomplish by using Unity Catalog for Data Quality.

Thanks,

Laksh

Re: Real time data quality validation (Streaming data ingestion)

Anonymous — Tue, 21 Mar 2023 06:53:25 GMT

Hi @arun laksh

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: Real time data quality validation (Streaming data ingestion)

Debayan — Thu, 02 Mar 2023 06:43:06 GMT

Hi @arun laksh , could you please refer to https://docs.databricks.com/structured-streaming/unity-catalog.html and let us know if this helps.