abaudit
Open source Python library that audits the statistical validity of A/B test results. Most experimentation tools tell you whether a result is significant. abaudit tells you whether to trust it.
Built on the Ioannidis (2005) Positive Predictive Value framework, the library detects Sample Ratio Mismatch, optional stopping inflation, multiple metric fishing, and data fabrication, then computes a composite bias score and generates a self-contained HTML audit report.
- Pip-installable · CI/CD via GitHub Actions · Published on PyPI
- 184 tests · 99% coverage · tested on Python 3.9 – 3.12
- Modules:
validity,design,runtime,report
- On PyPI
- 184 tests
- 99% coverage