Bayesian Experimentation Triptych
probability
pymc
experimentation
a/b testing
Abstract
Looping through experiment design via Assurance, through Sensitivity Analysis and Meta-analysis
Experiment Design Workflow in PyMC
This project added three new demonstrations to the PyMC Examples site, staged above as a Boschian triptych of the experimentation lifecycle. The learning loop, plan, interpret, synthesize, then plan again with what you learned, is central to the Bayesian experimentation workflow.
- → Assurance Planning via Simulation
- → Sensitivity Analysis for Unmeasured Confounding
- → Multiple Experiments & Bayesian Meta-analysis
- …
- → Assurance Planning via Simulation
Each notebook jumps off a theme in the FDA’s guidance on how to plan, probe and consolidate information derived from an experimentation programme.
They will ideally be read as a learning loop.
