abaudit library

abaudit

Open source Python library that audits the statistical validity of A/B test results. Most experimentation tools tell you whether a result is significant. abaudit tells you whether to trust it.

Built on the Ioannidis (2005) Positive Predictive Value framework, the library detects Sample Ratio Mismatch, optional stopping inflation, multiple metric fishing, and data fabrication, then computes a composite bias score and generates a self-contained HTML audit report.

  • Pip-installable · CI/CD via GitHub Actions · Published on PyPI
  • 184 tests · 99% coverage · tested on Python 3.9 – 3.12
  • Modules: validity, design, runtime, report
  • On PyPI
  • 184 tests
  • 99% coverage

BA Host Advisor

Multi-agent advisor for short-term rental hosts on real Inside Airbnb data. A LangGraph supervisor routes questions to SQL, RAG and cache-augmented agents in parallel.

  • Eval-gated CI
  • CAG vs RAG benchmark
  • Tracing + Prometheus

Loan Intent Classification

Multi-label intent classification of Spanish free text from SME credit applications, served as an API with a Streamlit demo.

  • Test macro F1 0.732
  • +0.043 over baseline
  • 95% CIs

Factored AI & Data Hackathon 2026

Card and account inquiries assistant with step-up OTP verification, handoff to human agents and a full audit trail. Team project.

My role: TODO

Research notes