Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-13-2025 01:36 AM
Hello,
When evaluating data quality frameworks for PySpark pipelines, I came across DQX. I noticed it's available on PyPI (databricks-labs-dqx) and GitHub, which is great for accessibility.
However, I'm trying to understand the licensing requirements. The LICENSE information on PyPI mentions it as "Other/Proprietary", and the license file on GitHub states that the materials can only be used "in connection with your use of the Databricks Services."
I would like to understand,
- Does this mean DQX requires a Databricks platform agreement even when used in other Spark environments (e.g., AWS EMR, Dataproc, or standalone Spark clusters)?
- If a Databricks agreement is required, are there specific licensing options for using Labs projects outside the Databricks platform?
- Are there any recommended alternatives for data quality validation in non-Databricks PySpark environments that you'd suggest?
I want to ensure we're compliant with all licensing requirements before adopting any framework. Any clarification would be greatly appreciated.
Thanks in advance!
Kiran