Document Type: Framework
Status: Active
Version: v1.0
Authority: Experimentation Brain
Applies To: Experimentation Brain, Affiliate Brain, Data Brain, Finance Brain, Research Brain, HeadOffice
Parent: Experimentation Brain Canon
Last Reviewed: 2026-04-25
Purpose
The Experimentation Brain Long-Term Impact Framework defines how MWMS evaluates the delayed and persistent effects of decisions, tests, and changes over time.
Its purpose is to ensure MWMS does not rely solely on short-term results when evaluating performance.
Many decisions produce effects that extend beyond the initial test window.
This framework ensures MWMS captures and evaluates those effects before scaling, repeating, or institutionalising changes.
Core Principle
Short-term results do not represent full impact.
A decision that performs well in the short term may:
• degrade performance later
• weaken customer quality
• create dependency on incentives
• damage future profitability
A decision that appears weak initially may:
• strengthen over time
• improve customer quality
• produce compounding gains
MWMS must evaluate both immediate and delayed effects.
Definition
Long-term impact refers to the sustained effect of a decision beyond the initial observation period.
This includes:
• behavioural changes
• customer quality changes
• offer performance decay
• creative fatigue
• funnel degradation
• profitability shifts
Core Question
This framework answers:
👉 What happens after the test result?
Impact Layers
1. Behavioural Persistence Layer
Tracks whether behavioural change continues after the test period.
Signals may include:
• repeat engagement
• sustained conversion behaviour
• continued progression through funnel stages
Purpose
• identify durable behaviour vs temporary response
• detect shallow wins
2. Customer Quality Impact Layer
Tracks how decisions affect customer quality over time.
Signals may include:
• repeat purchase behaviour
• spend per customer
• engagement stability
• discount dependency
Purpose
• detect quality degradation
• protect long-term value
3. Offer Stability Layer
Tracks whether an offer remains stable after scaling or changes.
Signals may include:
• conversion rate consistency
• engagement consistency
• performance across segments
Purpose
• detect offer fatigue
• detect structural weakness
4. Creative Durability Layer
Tracks whether creative performance persists over time.
Signals may include:
• declining CTR
• declining engagement
• audience fatigue
• repeated exposure effects
Purpose
• identify creative decay
• avoid over-reliance on short-term winners
5. Funnel Integrity Layer
Tracks whether funnel performance remains stable after changes.
Signals may include:
• drop-off changes
• behavioural friction
• conversion stability
• stage progression
Purpose
• detect hidden funnel damage
• identify delayed friction
6. Profitability Layer
Tracks whether decisions affect profit over time.
Signals may include:
• margin changes
• discount dependency
• customer value trends
• cost of acquisition changes
Purpose
• prevent revenue growth masking profit decline
• protect capital
Impact Time Horizons
Short-Term
0–30 days
Used for:
• initial signal detection
• early validation
Mid-Term
30–90 days
Used for:
• stability assessment
• repeatability validation
Long-Term
90+ days
Used for:
• durability evaluation
• strategic decision validation
Evaluation Workflow
Step 1 — Capture Initial Result
Record:
• test classification
• expected outcome
• actual outcome
• variance
Step 2 — Define Impact Monitoring Window
Assign:
• time horizon
• key metrics
• monitoring frequency
Step 3 — Track Impact Layers
Monitor:
• behaviour
• customer quality
• offer performance
• creative durability
• funnel performance
• profitability
Step 4 — Compare Against Baseline
Evaluate:
• whether performance is stable
• whether performance improves or degrades
• whether variance increases
Step 5 — Validate Data
Data Brain must confirm:
• signal integrity
• measurement reliability
• segmentation validity
Step 6 — Classify Impact
Classify outcome as:
• durable positive
• temporary positive
• neutral
• delayed negative
• long-term negative
Step 7 — Feed Back Into System
Route findings to:
• Affiliate Brain (offer decisions)
• Experimentation Brain (future tests)
• Research Brain (pattern recognition)
• Finance Brain (capital control)
• HeadOffice (strategic narrative)
Decision Rules
MWMS must not:
• scale based only on short-term results
• institutionalise changes without durability validation
• ignore delayed negative effects
MWMS may:
• scale cautiously with monitoring
• repeat tests to validate durability
• roll back changes if long-term impact is negative
Scaling Control Rule
Scaling must be limited if:
• long-term impact is unknown
• customer quality declines
• performance becomes unstable
• profitability weakens
Scaling may increase if:
• impact is durable
• customer quality improves
• performance remains stable
• profit logic holds
Cross Brain Use
Experimentation Brain
Owns long-term impact evaluation.
Data Brain
Validates signals and tracks consistency.
Affiliate Brain
Uses long-term signals to judge offer viability.
Finance Brain
Uses impact data to control capital allocation.
Research Brain
Stores long-term patterns for reuse.
HeadOffice
Uses long-term impact to guide strategy.
Relationship To Other Frameworks
This framework connects to:
• Experimentation Brain Test Lifecycle Model
• Experimentation Brain Test Interpretation Discipline
• Experimentation Brain Test Result And Decision Workflow
• Data Brain Customer Quality Tracking Framework
• Data Brain Performance Decomposition Framework
• MWMS Promotion Impact Framework
• Affiliate Brain Offer Health Monitoring Framework
Failure Modes Prevented
This framework prevents:
• scaling short-term winners that fail later
• ignoring delayed negative effects
• overvaluing early results
• missing offer or creative decay
• damaging customer quality unknowingly
• sacrificing profit for short-term gain
Drift Protection
The system must prevent:
• short-term results replacing long-term evaluation
• delayed effects being ignored
• monitoring windows being skipped
• incomplete impact analysis
• repeated mistakes due to missing long-term insight
Architectural Intent
Long-Term Impact Framework ensures MWMS decisions are evaluated beyond the initial test window.
It transforms MWMS from:
👉 short-term optimisation
to
👉 durable system optimisation
Final Rule
If long-term impact is unknown:
→ scaling must remain controlled
Change Log
Version: v1.0
Date: 2026-04-25
Author: Experimentation Brain / HeadOffice
Change
Initial creation of Long-Term Impact Framework based on transactional analysis insight that business decisions produce delayed effects across time.
Change Impact Declaration
Pages Created:
Experimentation Brain Long-Term Impact Framework
Pages Updated:
None
Pages Deprecated:
None
Registries Requiring Update:
Experimentation Brain Architecture
MWMS Architecture Registry
Canon Version Update Required:
No
Change Log Entry Required:
Yes