Document Type: Framework
Status: Structural
Version: v1.0
Authority: HeadOffice
Applies To: Experimentation Brain, Data Brain, Affiliate Brain, Ads Brain, Research Brain
Parent: Experimentation Brain
Last Reviewed: 2026-04-22
Purpose
The Measurement Hypothesis Mapping Framework defines how MWMS connects behavioural hypotheses to measurable signals.
It ensures experiments are supported by measurement structures capable of producing reliable evidence.
Experiments must not begin until measurable evaluation criteria exist.
This framework prevents:
untestable hypotheses
ambiguous experiment outcomes
reliance on vanity metrics
unclear experiment success criteria
weak causal interpretation
unreliable learning conclusions
Experiments must be measurable before they are executable.
Scope
This framework governs:
experiment measurement design
hypothesis-to-signal mapping
primary success metric definition
supporting diagnostic signal definition
experiment evaluation structure
measurement readiness checks
signal sufficiency for experimentation
This framework applies to:
conversion optimisation experiments
advertising experiments
offer experiments
funnel experiments
landing page experiments
hook testing experiments
behavioural experiments
pricing experiments
message testing experiments
Core Principle
A hypothesis must specify measurable evidence before an experiment can be executed.
Experiments without defined measurement criteria cannot produce reliable learning.
Measurement clarity determines experiment validity.
Hypothesis Structure Requirement
Each hypothesis must define:
proposed behavioural change
expected outcome
expected direction of change
affected user segment
expected mechanism of influence
expected measurable effect
Example structure:
Changing the headline to emphasise speed will increase click-through rate because urgency increases perceived value.
Each hypothesis must produce measurable implications.
Measurement Requirement for Hypotheses
Each hypothesis must define:
Primary Evaluation Signal
Supporting Diagnostic Signals
Risk Signals
Segment Interpretation Signals
Measurement structure must exist prior to experiment launch.
Primary Evaluation Signal
Primary signals define experiment success criteria.
Primary signals represent the core behavioural outcome the experiment aims to influence.
Examples:
conversion rate
click-through rate
revenue per visitor
earnings per click
cost per acquisition
lead generation rate
funnel progression rate
Primary signals must:
align with business objectives
align with decision thresholds
remain stable across experiments
allow statistical interpretation
Experiments must define success thresholds before execution.
Supporting Diagnostic Signals
Supporting signals help interpret behavioural mechanisms.
Supporting signals explain why a change produced a result.
Examples:
scroll depth
time on page
video engagement
form interaction behaviour
checkout step progression
session engagement metrics
bounce behaviour
Diagnostic signals improve learning depth.
Diagnostic signals improve experiment interpretation.
Risk Signals
Risk signals identify unintended negative outcomes.
Examples:
increase in bounce rate
decrease in session duration
decrease in downstream conversion
increase in refund rate
increase in unsubscribe rate
increase in complaint rate
increase in support requests
Risk signals prevent optimisation that produces hidden harm.
Risk signals protect system stability.
Segment Interpretation Signals
Segment-level signals help determine whether effects vary across audiences.
Examples:
device type differences
traffic source differences
campaign angle differences
new vs returning users
geographic variation
audience segment variation
Segment interpretation improves scaling decisions.
Measurement Sufficiency Requirement
Experiments require sufficient measurement clarity to produce interpretable results.
Measurement must allow:
clear outcome evaluation
behavioural interpretation
causal reasoning
statistical evaluation
decision confidence
Insufficient measurement reduces experiment value.
Signal Mapping Structure
Hypotheses must map clearly to signals.
Hypothesis
→ Behavioural Mechanism
→ Observable Behaviour
→ Signal Definition
→ Metric Definition
→ Experiment Evaluation
Each hypothesis must identify expected behavioural evidence.
Experiment Signal Hierarchy
Each experiment must define:
Primary Metric
Secondary Metrics
Diagnostic Metrics
Risk Metrics
Segment Metrics
Metric hierarchy ensures structured interpretation.
Metric hierarchy prevents misinterpretation.
Measurement Alignment Requirement
Experiment signals must align with signal definitions used elsewhere in MWMS.
Signal reuse improves:
comparability across experiments
consistency across campaigns
consistency across offers
continuity of learning
Signal fragmentation reduces knowledge accumulation.
Pre-Experiment Measurement Validation
Before experiment execution, measurement readiness must be confirmed.
Measurement readiness includes:
signal definitions documented
event structure defined
parameters defined
conversion definitions defined
validation logic defined
Measurement must be testable prior to experiment launch.
Relationship to Data Brain
Data Brain defines:
signal structure
parameter definitions
event definitions
Data Brain ensures signals are technically measurable.
Experimentation Brain ensures signals are behaviourally meaningful.
Relationship to Affiliate Brain
Affiliate experiments require:
conversion evaluation signals
funnel progression signals
engagement signals
signal clarity ensures accurate offer decisions.
Relationship to Ads Brain
Ad experiments require:
click-through signals
engagement signals
conversion signals
cost signals
consistent signals enable optimisation velocity.
Relationship to Research Brain
Research Brain contributes:
behavioural insight
hypothesis inspiration
mechanism interpretation
behavioural pattern analysis
Research improves hypothesis quality.
Experiment Readiness Criteria
An experiment is measurement-ready when:
primary success signal defined
signal collection method defined
diagnostic signals defined
decision thresholds defined
validation method defined
If measurement readiness is incomplete, experiment execution must pause.
Drift Prevention
Measurement drift occurs when:
primary metrics change across experiments
diagnostic signals change meaning
event definitions change inconsistently
conversion definitions change without documentation
Drift reduces comparability of results.
Drift reduces reliability of learning.
Consistent measurement definitions improve knowledge accumulation.
Minimum Viable Experiment Signal Set
Experiments should initially define:
primary success signal
minimum supporting diagnostic signals
minimum risk signals
Additional signals may be added when justified.
Minimal signal sets enable faster experimentation cycles.
Change Log
Version: v1.0
Date: 2026-04-22
Author: HeadOffice
Change:
Initial creation.
Defines structured mapping between hypotheses and measurable signals.
Improves experiment reliability and interpretability across MWMS.