Document Type: Standard
Status: Active
Version: v1.2
Authority: HeadOffice
Applies To: Experimentation Brain, Ads Brain, Affiliate Brain, Research Brain, Finance Brain
Parent: Experimentation Brain
Last Reviewed: 2026-04-22
Purpose
Experimentation Brain Evidence Hierarchy defines how different types of evidence are weighted when interpreting experimental results.
Not all signals carry equal reliability.
Some signals are more stable, more repeatable, and more decision-relevant than others.
Evidence hierarchy ensures MWMS does not overvalue weak indicators or undervalue strong indicators.
Evidence weighting improves decision consistency.
Evidence weighting improves learning reliability.
Evidence weighting improves scaling safety.
Evidence weighting improves capital discipline.
Evidence weighting improves experimentation stability.
Evidence weighting improves confidence progression accuracy.
Core Principle
Evidence should influence decisions in proportion to its reliability.
Higher-quality evidence:
reduces uncertainty
improves confidence accuracy
supports safer scaling decisions
supports stronger learning continuity
Lower-quality evidence:
may still provide insight
but should not dominate decision influence
Evidence hierarchy protects decision discipline.
Evidence strength must reflect measurement strength.
Weak measurement structures weaken evidence strength.
Reliable measurement strengthens evidence quality.
Role Inside MWMS Ecosystem
Evidence hierarchy connects:
Experimentation Brain
Ads Brain
Affiliate Brain
Finance Brain
Research Brain
HeadOffice
It ensures signal interpretation reflects structural evidence strength.
Evidence hierarchy supports:
confidence progression discipline
capital exposure discipline
signal reliability awareness
learning loop quality
Consistent evidence weighting improves system intelligence.
Consistent evidence weighting improves optimisation stability.
Measurement Dependency Principle
Evidence reliability depends on measurement reliability.
Evidence strength cannot exceed signal integrity.
Poor signal design weakens evidence strength.
Unstable measurement structures reduce interpretability.
Reliable evidence requires:
stable event definitions
consistent parameter structure
consistent conversion definitions
reliable signal transmission
stable measurement architecture
validated signal chain continuity
Measurement quality sets the upper limit of evidence reliability.
Signal instability reduces evidence confidence.
Evidence Reliability Factors
Evidence reliability may be influenced by:
signal consistency
signal stability
environmental robustness
sample relevance
structural clarity
repeatability
measurement architecture quality
signal design clarity
data layer reliability
parameter consistency
event definition stability
multi-layer validation confirmation
Evidence demonstrating stability across conditions is typically more reliable.
Evidence based on unstable signals requires caution.
Signal validation increases confidence in evidence interpretation.
Measurement Structure Strength
Evidence strength improves when signals are:
clearly defined
structurally consistent
consistently transmitted
interpreted consistently
reusable across experiments
comparable across time
validated across measurement layers
Measurement structure stability improves evidence comparability.
Inconsistent signal structure weakens evidence interpretation.
Stable measurement architecture strengthens confidence progression discipline.
Relative Evidence Strength
Evidence can vary in structural reliability depending on:
consistency of observation
repeatability of results
sensitivity to environmental variation
clarity of causal relationship
alignment with prior validated signals
stability of signal definitions
consistency of measurement structure
validation completeness
signal routing stability
Stronger evidence supports stronger confidence progression.
Weaker evidence should influence decisions cautiously.
Signal clarity improves interpretation stability.
Evidence Context Awareness
Evidence should be interpreted relative to its context.
Context may include:
test environment
platform behaviour
audience characteristics
creative structure
offer structure
timing conditions
measurement environment
tracking configuration
signal validation completeness
implementation architecture
Context improves interpretation accuracy.
Evidence without measurement context may produce misleading conclusions.
Measurement architecture influences interpretation confidence.
Multi-Layer Validation Influence
Evidence confidence increases when signals are validated across multiple layers:
data layer structure
tag execution confirmation
network transmission integrity
analytics interpretation layer
reporting consistency
Signals validated across multiple layers demonstrate higher structural reliability.
Single-layer signals require cautious interpretation weighting.
Validated signals increase confidence progression stability.
Interaction with Signal Integrity Framework
Signal Integrity Framework defines whether signals accurately reflect behaviour.
Evidence hierarchy assumes signal integrity has been evaluated.
If signal integrity is uncertain, evidence strength must be discounted.
Signal integrity influences:
confidence progression
learning reliability
decision stability
Signal instability reduces evidence reliability.
Stable signal architecture strengthens evidence strength.
Interaction with Measurement Strategy Framework
Measurement Strategy Framework defines:
what signals exist
why signals exist
how signals support decision-making
Evidence hierarchy evaluates how strongly those signals support decisions.
Evidence weighting depends on signal relevance.
Signals not tied to decision requirements should not heavily influence confidence progression.
Signal relevance influences evidence weight.
Decision alignment improves evidence usefulness.
Interaction with Signal Design Specification Framework
Signal Design Specification Framework defines:
event structure clarity
parameter structure clarity
conversion definition clarity
consistent signal definitions improve evidence comparability.
Signal ambiguity reduces evidence reliability.
Signal fragmentation weakens learning continuity.
Clear signal definitions improve interpretation reliability.
Interaction with Data Layer Architecture Framework
Data Layer Architecture Framework defines reliability of signal transmission.
Unstable data transmission reduces evidence reliability.
Stable data architecture improves:
signal continuity
experiment comparability
confidence progression reliability
Measurement infrastructure quality influences evidence strength.
Signal chain continuity strengthens evidence comparability.
Interaction with Signal Confidence Model
Signal confidence should reflect evidence quality.
Higher confidence requires stronger evidence.
Weak evidence should not produce strong confidence classification.
Confidence discipline depends on evidence discipline.
Confidence progression must reflect structural reliability of signals.
Confidence accuracy improves capital discipline.
Interaction with Test Interpretation Discipline
Interpretation discipline ensures evidence meaning is understood correctly.
Evidence hierarchy ensures evidence strength is weighted appropriately.
Strong interpretation depends on understanding both:
signal meaning
signal strength
measurement stability
validation completeness
interpretation clarity improves decision accuracy.
Interpretation discipline reduces false optimisation direction.
Interaction with Learning Loop Integrity
Learning Loop Integrity ensures evidence contributes to long-term knowledge accumulation.
Evidence hierarchy improves the quality of stored learning.
Higher-quality evidence produces more valuable long-term insight.
Learning quality depends on evidence quality.
Measurement consistency improves knowledge continuity.
Stable signals improve reusable learning value.
Interaction with Financial Signal Sensitivity
Financial exposure decisions should reflect evidence strength.
Weak evidence may justify exploratory testing.
Moderate evidence may justify controlled testing expansion.
Strong evidence may justify structured scaling.
Evidence hierarchy supports capital protection discipline.
Stronger evidence reduces capital risk.
Signal stability improves capital confidence.
Interaction with Ads Brain Performance Signals
Ads Brain produces performance metrics.
Evidence hierarchy ensures performance metrics are interpreted with appropriate weight.
Performance variation alone does not determine structural validity.
Evidence context influences signal reliability.
Measurement clarity improves interpretation of performance variation.
Signal validation improves performance confidence.
Interaction with Affiliate Brain Opportunity Evaluation
Affiliate Brain evaluates structural opportunity viability.
Evidence hierarchy ensures opportunity progression decisions reflect reliable signal strength.
Weak evidence should not justify strong opportunity classification.
Strong evidence strengthens opportunity confidence.
Measurement clarity improves opportunity evaluation accuracy.
Signal continuity improves opportunity comparability.
Evidence Consistency Considerations
Evidence reliability improves when observations demonstrate consistency across:
multiple variations
multiple audiences
multiple environments
multiple time windows
multiple signal structures
multiple validation layers
Consistent evidence strengthens confidence progression.
Inconsistent evidence requires caution.
Signal stability strengthens confidence reliability.
Evidence Stability Considerations
Stable signals tend to:
demonstrate similar behaviour under varied conditions
maintain directionality across environments
produce repeatable patterns
maintain consistent measurement structure
maintain consistent parameter interpretation
Stable evidence supports structured decision progression.
Unstable evidence requires further validation.
Signal volatility reduces evidence confidence weighting.
Structural Examples
Example A
A performance improvement appears only in a single variation under narrow conditions.
Interpretation:
evidence may be exploratory.
Confidence progression should remain limited.
Further validation recommended.
Example B
Performance improvements appear across multiple variations and audiences.
Interpretation:
evidence strength may support confidence progression.
Further structured validation may be appropriate.
Confidence progression may increase cautiously.
Example C
Signals fluctuate significantly under minor environmental changes.
Interpretation:
evidence stability may be weak.
Measurement structure should be reviewed.
Additional testing may improve signal clarity.
Signal instability suggests structural review may be required.
Out of Scope
This standard does not define:
exact statistical thresholds
specific platform metrics
specific campaign settings
specific test budgets
specific sample sizes
specific test durations
These belong to operational layers.
Evidence hierarchy defines structural weighting discipline.
Structural Summary
Experimentation Brain Evidence Hierarchy ensures MWMS decisions reflect appropriate weighting of signal strength.
It improves:
confidence accuracy
capital protection discipline
learning quality
decision stability
measurement discipline awareness
Stronger evidence improves safer scaling behaviour.
Stronger measurement improves stronger evidence.
Stronger signal architecture improves learning reliability.
Change Log
2026-04-22
Version: v1.2
Enhancement:
added multi-layer validation weighting influence
added signal chain stability considerations
added validation completeness influence on evidence strength
improved alignment with:
Signal Integrity Framework
Signal Design Specification Framework
Measurement Strategy Framework
Data Layer Architecture Framework
Debugging and Validation Framework