Document Type: Canon
Status: Canon
Version: v1.7
Authority: MWMS HeadOffice
Applies To: Experimentation Brain statistical governance, methodological integrity, and cross-MWMS experiment standards
Parent: Brains
Last Reviewed: 2026-04-18
Purpose
The Experimentation Brain governs statistical and methodological integrity across MWMS.
Its purpose is to ensure that:
• all experiments follow statistical discipline
• all tests meet minimum methodological standards
• false positives are controlled
• measurement integrity is protected
• business cases are calculated before scaling
• insights are aggregated and institutionalized
It does not execute experiments.
It does not originate hypotheses.
It defines how experiments must be executed once intent is declared.
It ensures experimentation produces trustworthy institutional knowledge rather than fragmented or biased learning.
Scope
This canon applies to:
• A/B testing methodology across MWMS
• statistical thresholds and power requirements
• duration and sequencing rules
• peeking policies and SRM standards
• KPI selection discipline
• business-case validation before scale
• incrementality testing where required
• meta-study aggregation and institutional learning
• cross-brain experiment governance
• growth experimentation process discipline
• experiment classification standards
• experiment documentation sufficiency rules
This document governs the constitutional role and mandatory standards of Experimentation Brain.
It does not govern:
• offer approval
• capital allocation
• runtime system monitoring
• ad execution
• capital risk classification
• marketing strategy creation
Those remain governed by Affiliate Brain, Finance Brain, SIT Brain, HeadOffice, and related system canons.
Core Purpose
Experimentation Brain exists to protect the validity of learning inside MWMS.
It is the methodological governor of experiments, not the business originator of experiments.
It ensures experimentation produces trustworthy institutional knowledge rather than noisy, ego-driven, or statistically weak conclusions.
Experimentation exists to improve decision quality through reliable signal generation.
Position in MWMS Architecture
Experimentation Brain is a governance peer to:
• SIT Brain
• Finance Brain
It supports:
• Affiliate Brain
• Ads Brain
• Product Brain
• Conversion Brain
• AI Business Systems Brain
• any future testing environment
Hierarchy:
HeadOffice
↳ Experimentation Brain
↳ SIT Brain
↳ Finance Brain
Experimentation Brain ensures consistent learning discipline across all execution environments.
Intent Alignment Requirement
Experimentation Brain may not initiate Phase 1 unless the MWMS opportunity lifecycle has progressed through:
• Research Signal
• Opportunity Queue
• Offer Intelligence Evaluation
• Affiliate Structural Evaluation
And the following conditions are satisfied:
• structured hypothesis declared
• capital risk classification defined
• lifecycle stage declared
• Velocity = YES
• research integrity status = OK
Experimentation validates statistical form.
It does not create business intent.
If Intent Gate is incomplete:
Output =
BLOCKED — Intent Misalignment
Growth Process Governance
Experimentation Brain governs the structured learning loop used across MWMS experimentation environments.
Core experimentation loop:
idea
prioritisation
test
analysis
learning
iteration
Each stage contributes to cumulative learning.
Skipping stages reduces institutional intelligence quality.
The Growth Process ensures:
tests are intentional
learning accumulates
hypotheses improve over time
experiments align with Growth Levers
knowledge compounds across lifecycle stages
Experimentation Brain governs the integrity of this loop.
It does not control idea generation itself.
Experiment Classification Standard
Experimentation Brain distinguishes between different types of change to preserve clarity in learning interpretation.
Growth Experiment
A growth experiment is a structured test designed to generate decision-relevant learning.
Characteristics:
clear hypothesis
defined expected behavioural change
measurable success condition
meaningful potential impact
relevant to active Growth Lever
Growth experiments typically involve meaningful change magnitude.
Examples:
new messaging angle
new landing page structure
new funnel logic
new offer positioning
new onboarding flow
new pricing structure
Growth experiments aim to reduce uncertainty regarding meaningful system constraints.
Optimisation
Optimisation represents incremental improvement applied to an existing asset or process.
Optimisations typically involve:
smaller adjustments
lower structural uncertainty
incremental improvement potential
Examples:
refining copy clarity
adjusting targeting parameters
adding FAQ clarification
improving formatting clarity
reducing friction in micro-interactions
Optimisations may not always require full experimental structure.
Optimisation documentation depth should reflect expected learning value.
Over-documentation may degrade experimentation velocity.
A/B Test
An A/B test is a specific experimental method.
It compares two variants simultaneously.
Example:
headline A vs headline B
layout A vs layout B
CTA A vs CTA B
Not all experiments require A/B structure.
Other valid methods include:
time-based testing
sequential testing
staged rollout testing
holdout testing
Experiment classification should not constrain appropriate methodology.
Relationship to Growth Lever Structure
Experimentation operates within structured Growth Levers.
Structure:
North Star Metric
↓
Growth Lever
↓
Theme
↓
Experiment
Growth Levers define:
which constraint is prioritised.
Themes define:
which behavioural mechanisms are investigated.
Experiments generate:
evidence regarding behavioural response.
Experimentation Brain ensures signal validity across this structure.
Experiment Lifecycle Standard (Mandatory)
Every experiment must follow structured lifecycle discipline.
Phase 1 — Hypothesis Validation
• hypothesis supplied by Affiliate Brain or relevant execution Brain
• primary KPI confirmed
• secondary KPIs defined
• expected uplift validated
• risk threshold confirmed
• minimal detectable effect defined
Experimentation validates structure, not business narrative.
Phase 2 — Power & Duration Calculation
• minimum sample size calculated
• power ≥ 80% (default rule)
• duration estimate documented
• traffic validation completed
Phase 3 — Pre-Test Integrity Checks
• funnel measurement verified
• tool vs analytics traffic reconciled
• tracking code verified
• variation allocation confirmed
Phase 4 — Execution Discipline
• no peeking before minimum sample threshold
• no mid-test KPI switching
• no audience reshaping mid-test
• no creative modification during run
Campaign structure must remain fixed during experiment execution.
Phase 5 — Outcome Classification
• Win
• Loss
• Inconclusive
• False positive risk flagged
Phase 6 — Business Case Validation
• expected value calculated
• implementation cost included
• risk-adjusted ROI calculated
Finance Brain gate approval is required before scaling.
Incrementality Testing Protocol
Certain marketing activities require validation of true incremental impact, not just relative performance between variants.
Standard A/B testing determines which variant performs better.
Incrementality testing determines whether an activity generates additional outcomes that would not have occurred without intervention.
Incrementality tests may be required when evaluating:
• brand advertising
• upper-funnel acquisition campaigns
• retargeting systems
• affiliate traffic sources
• channel expansion initiatives
• algorithmic bidding systems
• CRM and lifecycle marketing programs
Incrementality testing methods may include:
• holdout groups
• ghost ads
• geo-based experimentation
• audience exclusion experiments
• time-based counterfactual testing
Scaling decisions must consider incremental value, not only attribution.
Measurement Integrity Protocol
Experimentation Brain enforces:
• funnel completeness audits
• analytics vs test-tool traffic comparison
• code execution rate monitoring
• client vs server-side test evaluation
• first-party data preference
• ITP / cookie limitation mitigation review
If measurement gap > 10% → test flagged
If measurement gap > 20% → scaling blocked
Measurement integrity protects decision validity.
False Positive Protection Rules
Mandatory rules:
• minimum 80% power
• no stopping early without predefined sequential rule
• SRM monitoring required
• false positive estimation included in reporting
Probability of being best does not equal automatic scale.
Scaling requires expected value validation.
Meta Study & Behavioral Intelligence Layer
All experiments must be logged with:
• funnel stage
• traffic source
• offer type
• persuasion technique used
• psychological mechanism
• uplift %
• statistical confidence
• sample size
• business outcome
Experimentation Brain maintains:
• insight database
• cross-test aggregation
• persuasion pattern detection
• funnel-stage effectiveness mapping
• long-term behavioral model development
Institutional learning strengthens predictive capability.
Prediction Validation Rule
All experiments must produce a forward prediction that can be validated against future performance.
Before scaling approval, experiment report must include:
• expected uplift projection
• confidence range of outcome
• forecast impact on business metrics
After implementation:
predicted uplift vs actual performance must be evaluated.
Variance informs model refinement.
Persistent prediction error may trigger:
• model recalibration
• experiment design review
• insight confidence downgrade
Prediction validation ensures learning compounds accurately.
Scaling Governance Model
When experimentation volume increases:
execution may distribute across operational teams.
However:
statistical standards remain centralized.
No department may run experiments outside this framework.
Governance consistency protects learning quality.
Interaction With SIT Brain
SIT monitors:
• runtime compliance
• logging
• data integrity
• rule violation detection
Experimentation Brain defines:
• statistical rules
• integrity thresholds
SIT enforces compliance.
Experimentation governs methodology.
Interaction With Finance Brain
Experimentation produces:
• statistical outcome
• expected uplift
• risk-adjusted projection
Finance Brain determines:
• capital allocation
• scaling approval
• budget expansion
No scaling occurs without Finance approval.
Interaction With Affiliate Brain
Affiliate Brain defines:
• hypothesis
• opportunity context
• testing intent
Experimentation Brain validates:
• statistical design
• sample requirements
• execution discipline
Affiliate Brain does not override statistical methodology.
Escalation Rule
If any of the following occur:
• statistical integrity violated
• measurement compromised
• peeking detected
• KPI switching detected
• intent misalignment detected
SIT escalates to HeadOffice.
HeadOffice may:
• invalidate test
• freeze scaling
• trigger audit
Canon Lock
Experimentation standards may only be modified by:
• HeadOffice approval
• documentation
• version update
No operational brain may override Experimentation rules.
Final Rule
No experiment inside MWMS is valid unless it satisfies methodological discipline, measurement integrity, and governance alignment.
Experimentation Brain protects the credibility of learning across the ecosystem.
Drift Protection
The system must prevent:
• experiments being treated as valid without statistical discipline
• business enthusiasm overriding experiment form
• scaling decisions being made from weak or contaminated data
• departments creating local testing standards outside central governance
• incrementality-sensitive channels being evaluated with incomplete logic
• institutional learning degrading into disconnected test anecdotes
Experimentation governance must remain centralized, consistent, and enforceable.
Architectural Intent
Experimentation Brain exists to make statistical and methodological integrity a permanent governing layer inside MWMS.
Its role is to ensure that all Brains can test, learn, and scale within a common discipline so institutional knowledge compounds on valid evidence rather than noise, bias, or convenience.
Change Log
Version: v1.7
Date: 2026-04-18
Author: MWMS HeadOffice
Integrated Growth Process governance structure.
Added clarification between:
growth experiments
optimisations
A/B tests
Aligned experimentation lifecycle with Growth Lever structure.
Added documentation sufficiency discipline to prevent over-documentation noise.
Maintained statistical authority boundaries.
No change to methodological integrity standards.
Structural coherence preserved.